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PREFACE 


This  Note  describes  research  efforts  carried  out  under  the 
Project  AIR  FORCE -sponsored  project  on  Tactical  Air  Command  and 
Control.  A  major  goal  of  the  project  is  to  develop  general 
methodology  for  evaluating  tactical  air  command  and  control.  One 
approach  is  to  combine  appropriately  computer  modeling  and  military 
exercises.  The  Note  discusses  design  considerations  for  a  Computer- 
Aided  Exercise  Facility  for  that  purpose. 


SUMMARY 


The  purpose  of  this  Note  is  to  describe  a  concept  and  general 
design  for  a  Computer  Aided  Exercise  Facility  (CAEF)  for  evaluating 
tactical  air  command  and  control.  The  CAEF  would  capitalize  on  and 
synthesize  the  strong  points  of  both  command  and  control  exercises  and 
computerized  conflict  modeling. 

Computerized  combat  models  are  capable  of  simulating  complex 
combat  interactions  between  opposing  military  forces,  but  are  unable 
to  deal  adequately  with  the  complexities  of  the  human  decision 
processes  directing  those  forces.  Command  and  control  exercises,  on 
the  other  hand,  incorporate  this  human  element  directly.  However, 
today's  manually  run  training  exercises  are  of  necessity  pre-scripted 
and  limited  in  their  representation  of  conflict  by  calculation  and 
bookkeeping  limitations.  Hence,  the  weakness  in  each  technique  is  the 
strength  in  the  other,  and  their  appropriate  combination  should 
provide  a  powerful  mechanism  for  the  study  of  the  command  and  control 
process.  Such  a  facility  would  also  provide  far  more  realistic  and 
challenging  training  than  is  possible  with  current  manually  supported 
exercises . 

The  design  suggested  here  has  three  major  parts--the  C2 
Components  being  exercised;  an  Exercise  Driver  which  simulates  the 
conflict  environment  and  runs  the  exercise;  and  an  Evaluation  Unit 
responsible  for  formulating  and  conducting  the  evaluation.  Each  of 
these  would  be  integrated  man/machine  units  composed  of  both  humans 
and  computer  aids. 
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The  C2  Components  in  any  given  exercise  would  consist  of  the 
tactical  air,  and  possibly  land  or  sea,  command  and  control  elements 
necessary  to  support  the  exercise  and  evaluation  objectives.  During 
an  exercise,  these  components  would  interface  directly  and  solely  with 
the  Exercise  Driver,  which  would  represent  the  entire  conflict 
environment  and  react  to  command  and  control  actions  taken.  The 
Evaluation  Unit  would  be  responsible  for  preparing  an  evaluation 
design  prior  to  the  exercise,  overseeing  the  conduct  of  the  exercise 
and  collecting  necessary  data  to  assure  the  evaluation  goals  are  met, 
and  performing  the  evaluation  analyses  following  the  exercise. 

The  Exercise  Driver  is  the  core  of  the  evaluation  exercise.  It 
must  be  able  to  provide  a  simulated  conflict  environment  which  is 
sufficiently  realistic  and  challenging  to  encourage  realistic 
performance,  and  which  is  sufficiently  responsive  to  allow  that 
performance  to  affect  the  course  of  the  conflict  in  reasonable  ways. 
The  design  of  the  Exercise  Driver  must  differ  greatly  from  the  normal 
research  and  analysis  modeling  and  simulation  designs,  in  that  the 
processing  requirements,  manual  and  computer,  are  driven  primarily  by 
on-line  human  information  needs.  In  particular,  the  players  must  be 
serviced  with  information  which  is  in  both  form  and  substance  like 
that  they  would  receive  in  actual  practice.  Likewise,  the  staff  of 
the  Exercise  Driver  must  have  information  continually  to  perform  their 
tasks.  For  both  groups,  the  information  must  be  available  in  "live" 
time  rather  than  an  arbitrary  "simulation"  time  driven  by  processing 
requirements,  and  computerized  portions  of  the  simulation  must  be 
responsive  to  the  man-machine  interactions  which  dominate  this  mode  of 
operation. 


Other  design  considerations  include:  (1)  the  likely  need  for 
frequent  modification  to  meet  particular  evaluation  requirements;  (2) 
the  importance  of  systematic,  planned  evolutionary  development  of  the 
CAEF  at  a  pace  consistent  with  growing  experience  as  well  as  with 
technological  advancements;  and  (3)  the  availability  of  funds, 
facilities,  equipment  and  personnel.  These  factors  favor  a  highly 
modular  design  rather  than  a  design  based  on  a  single  very  large 
computer  model  which  does  everything. 

The  CAEF  would  be  an  ongoing  activity  in  which  learning  and 
evaluation  take  place  in  an  integrated  fashion  over  time,  rather  than 
as  a  disconnected  series  of  conceptually  distinct  one-time  exercises. 
The  most  obvious  parts  of  this  activity  take  place  during  the  run  of 
an  exercise,  but  a  great  deal  of  important  preparatory  activity  takes 
place  before  the  exercise,  and  much  of  the  analysis  and  synthesis  for 
the  final  evaluation  must  be  done  afterward.  The  Evaluation  Unit,  in 
particular,  will  have  an  ongoing  responsiblility  for  organizing  and 
maintaining  the  cumulative  database  derived  from  all  exercises  run  at 
the  CAEF,  for  performing  comparative  evaluations  using  that  database, 
and  for  evolving  the  CAEF  itself  and  developing  and  refining  concepts 
for  using  it  as  an  evaluative  tool.  The  CAEF  must  be  designed  with 
this  whole  process  in  mind,  and  not  focused  too  heavily  on  the 
exercise  period  per  se. 

The  purpose  of  the  CAEF  is  to  conduct  exercises  as  a  foundation 
for  the  understanding  and  evaluation  of  command  and  control  processes 
and  systems.  However,  the  evaluation  of  the  contribution  of  command 
and  control  to  the  combat  effectiveness  of  military  forces  is  an  ill- 
defined  and  poorly  understood  area,  in  need  of  considerable  conceptual 


-viii- 


and  intellectual  development.  Sufficient  development  of  concepts  and 
techniques  needs  to  be  completed  to  provide  a  basis  on  which  to 
establish  an  initial  capability,  but  the  CAEF,  and  in  particular  the 
Evaluation  Unit,  should  be  thought  of  in  part  as  a  testbed  within 
which  further  development  .can  take  place.  Design  of  the  CAEF,  then, 
is  not  a  straightforward  task  of  designing  an  organizational  structure 
and  procedure  to  perform  a  well-defined  and  well-understood  task. 
Rather,  it  requires  the  articulation  of  some  initial  exercise 
evaluation  concepts  and  a  design  permitting,  and  specifying  the 
responsibility  for,  self-evolution. 
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I.  INTRODUCTION 

One  of  the  most  difficult  areas  of  military  analysis  concerns 
understanding  the  role  of  command  and  control  and  the  evaluation  of 
its  contribution  to  the  overall  combat  effectiveness  of  a  military 
force.  It  is  hard  to  come  to  grips  with  the  issue  of  command  and 
control  effectiveness,  let  alone  to  address  it  in  a  hard  quantitative 
manner  as  is  commonly  done  in  related  areas  such  as  weapons  systems 
effectiveness.  The  purpose  of  this  note  is  to  describe  a  new  tool 
with  which  to  approach  this  problem,  a  Computer  Aided  Exercise 
Facility  (CAEF)  for  command  and  control  evaluation.  Such  a  facility, 
we  believe,  might  capitalize  on  and  synthesize  the  strong  points  of 
both  manual  exercises  and  computerized  combat  modeling,  allowing  each 
to  compensate  for  and  overcome  the  weaknesses  of  the  other. 

This  description  is  preliminary,  intended  to  serve  as  a  starting 
point  for  developing  the  concept  and  its  applicability. 

Computer  modeling  alone  has  been  used  for  evaluating  a  wide 
variety  of  systems,  ranging  from  social  to  mechanical,  and  has  been 
used  extensively  by  the  Air  Force  to  evaluate  military  systems  in  a 
conflict  environment.  Conflict  models  focused  on  engagement  outcomes 
have  evolved  over  the  years  to  a  point  where  they  have  achieved  a 
measure  of  face  validity  within  the  military  community.  A  similar 
evolution  of  command  and  control  modules  within  these  models  has  not 
occurred.  This  is  due  partly  to  previous  lack  of  interest,  and  partly 
to  the  complexity  of  representing  the  command  and  control  systems 
themselves.  But  perhaps  the  most  difficult  barrier  has  been  the 
general  lack  of  understanding  (at  least  in  the  form  necessary  to  allow 
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it  to  be  inserted  into  a  computer  model)  of  the  command  and  control 
decisionmaking  process --of  how  information  is  translated  into 
decisions.  The  computer  modeling  approach  does  not  now  adequately 
represent  the  basic  functions  of  the  command  and  control 
process--planning,  directing  and  controlling--and  there  appears  to  be 
no  workable  way  to  overcome  this  deficiency  within  the  current  state- 
of-the-art. 

Military  exercises  are  usually  conducted  for  procedural  training 
of  personnel  and  to  obtain  some  nonrigorous  insight  into  the 
capabilities  of  the  systems  and  forces  involved.  These  goals  can  be 
at  least  partially  accomplished  in  spite  of  severe  limitations  on  the 
dynamics  and  the  realism  of  the  conflict  arena  within  which  the 
exercise  supposedly  takes  place.  Live  force  play  is  grossly 
artificial  because  of  cost  and  safety  constraints  (a  fact  which  is 
obvious  to  all  participants,  including  those  involved  with  command  and 
control),  while  Command  Post  Exercises  using  constructive  forces  have 
their  own  sets  of  different  constraints.  Chief  among  these 
constraints  is  the  bookkeeping  required.  So  much  is  necessary  to 
provide  the  inputs  to  simulate  a  large-scale  conflict  that  in  current 
manual  exercises  most  of  the  play  must  be  pre-scripted  well  in 
advance.  Actions  taken  by  the  players  thus  generally  have  little 
effect  on  the  course  of  events.  This  renders  futile  any  attempt  to 
evaluate  command  and  control  capabilities  with  respect  to  their  effect 
on  force  employment,  and  casts  doubt  on  the  utility  of  measures  of 
internal  efficiency  of  the  system  components  and  even  limits  the 
utility  of  exercises  as  vehicles  for  training  command  and  control 
personnel . 
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Thus,  computer  modeling  and  manual  exercises  both  have  serious 
limitations  when  used  as  a  basis  for  the  study  and  evaluation  of 
command  and  control  processes.  The  approaches,  however,  are 
complementary,  in  that  each  possesses  the  potential  for  offsetting  the 
deficiencies  of  the  other.. 

The  principle  deficiency  of  the  computer  modeling  approach  is 
that  it  does  not  adequately  model  human  processes.  An  exercise 
framework  overcomes  this  limitation  by  incorporating  the  human  process 
within  the  exercise.  People  are  actually  present  making  decisions  and 
commanding  actions,  so  there  is  no  need  to  model  or  simulate  them. 

The  manual  exercise  is  limited  in  that  the  external 
environment--the  "war"  which  can  be  presented  to  the  human  players--is 
too  flat  and  unresponsive  either  to  truly  challenge  their 
decisionmaking  capabilities  or  to  provide  a  framework  within  which 
those  capabilities  can  be  tested  and  measured.  Computer  modeling  can 
overcome  these  limitations  by  providing  a  richer  and  more  reactive 
simulation  of  the  external  environment  than  can  a  control  team  working 
in  a  manual  mode.  The  combination  of  a  human  exercise  team  playing 
within  a  computer-driven  conflict  environment  should  thus  produce  a 
far  better  vehicle  for  the  study  of  command  and  control  processes  than 
does  either  of  its  components  alone. 

These  ideas  are  portrayed  conceptually  in  Fig.  1.  Real  combat  is 
a  complex  and  untidy  process,  as  indicated  by  the  rough  and  uneven 
shape  in  Fig.  la.  Command  and  control  is  a  complex  and  untidy  part  of 
that  process,  indicated  by  its  uneven  shape  within  the  combat  process. 
Combat  modeling  deals  with  representations  of  both  processes  in  which 
all  the  rough  edges  have  been  smoothed  off,  and  the  whole  thing  has 
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been  made  ouch  more  regular  and  orderly,  as  depicted  in  Fig.  lb.  For 
some  purposes,  such  as  the  study  of  the  effects  of  changes  in  weapons 
system  performance  parameters,  the  distortions  introduced  by  this 
smoothing  are  acceptable,  even  desirable.  But  this  is  frequently  not 
the  case  for  studying  the  command  and  control  process.  Too  much 
important  substance  may  be  removed,  limiting  the  amount  that  can  be 
learned  about  the  real  command  and  control  process  from  its 
emasculated  counterpart  in  the  combat  model. 


Fig.  1  — Command  and  control  in  real  and  artificial  conflict 
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Moving  in  the  other  direction,  exercise  command  and  control 
processes  retain  much  of  the  richness  and  complexity  of  the  real 
process,  because  the  same  types  of  people  performing  the  same  types  of 
activities  are  involved  in  both.  She  simulated  environment  within 
which  most  exercises  are  conducted,  however,  is  too  flat  and  one¬ 
dimensional  to  realistically  stress  command  and  control  and  provide 
measurable  feedback  about  how  it  performs  (Fig.  1c).  But  it  should  be 
possible  to  develop  a  computer-aided  exercise  methodology  in  which  the 
real  command  and  control  process,  with  all  its  complexities  and  rough 
edges,  is  embedded  in  and  interacts  with  a  man/machine  simulation  of 
an  external  combat  environment  of  sufficient  complexity  and 
responsiveness  to  provide  a  reasonable  basis  for  evaluation  of  command 
and  control  performance  and  effectiveness  (Fig.  Id). 

We  are  concerned  with  the  conceptual  design  of  a  CAEF  which  would 
integrate  a  human  command  and  control  process  with  a  computer-aided 
simulation  of  one  conflict  environment.  The  CAEF  would  provide  an 
environment  within  which  appropriate  portions  of  the  existing  command 
and  control  system  (or  alternatives  or  proposed  improvements  thereto) 
could  be  exercised  against  a  computer-aided  combat  simulation 
providing  a  degree  of  realism,  challenge  and  responsiveness  to  actions 
taken  not  otherwise  attainable  apart  from  actual  combat.  For 
evaluative  purposes,  the  CAEF  might  be  better  than  actual  combat.  The 
actions  taken  and  the  effects  of  various  actions  can  be  monitored  and 
analyzed  to  a  degree  impossible  in  actual  combat,  and  replications  and 
systematic  variations  would  be  possible  as  well. 

While  we  are  concerned  here  with  the  CAEF  as  an  evaluation  tool, 
it  should  be  noted  that  it  provides  important  training  capabilities  as 
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well.  The  sane  flatness  and  unresponsiveness  which  limits  the 
evaluative  value  of  manual  exercises  also  limits  their  training  value, 
and  may  sometimes  encourage  trainees  to  learn  the  wrong  lessons.  The 
kind  of  responsive  environment  needed  for  evaluation  should  provide 
players  with  a  much  more  realistic  and  challenging  training  experience 
if  operated  in  a  training  mode,  and  we  would  expect  the  CAEF  to  be 
used  for  both  purposes. 

The  CAEF  should  not  be  thought  of  as  a  conceptually  distinct 
series  of  one-shot  exercises,  but  as  an  ongoing  activity  in  the 
business  of  conducting  computer-aided  command  and  control  exercises 
and  evaluating  and  learning  from  the  results.  The  ongoing  nature  of 
the  activity  is  important  because  it  allows  learning  and  evaluation  to 
take  place  in  an  integrated  fashion  over  time,  and  the  evaluation 
function  must  be  performed  in  a  way  which  takes  advantage  of  this. 

This  has  significant  implications  for  the  design  and  operation  of  the 
CAEF. 

It  is  our  intent  to  explore  design  and  operating  concepts  and 
develop  a  feasible  preliminary  design  for  such  a  facility.  But  it  is 
not  our  intent  to  prescribe  structure  and  operations  in  detail.  We 
feel  that  must  evolve  over  time.  Our  preliminary  design  will  provide 
a  starting  point  to  be  fleshed  out  as  the  facility  itself  is 
implemented  incrementally.  To  accomplish  our  task,  we  give  some 
attention  to  issues  beyond  those  of  designing  and  running  computer- 
aided  exercises,  including  the  evaluation  process  itself,  the 
operation  of  an  ongoing  facility  and  the  organization  supporting  it. 


The  principal  activity  of  the  CAEF  is  to  conduct  computer-aided 
evaluation  exercises,  i.e.,  computer-aided  exercises  for  the  purpose 
of  evaluating  command  and  control  systems  and  processes.  The  most 
obvious  part  of  this  activity  takes  place  during  the  run  of  an 
exercise — while  the  players  are  at  the  facility  engaged  in  fighting  an 
imaginary  "war"  and  the  whole  place  has  an  active  combat  air  to  it. 

But  this  is  only  a  part  of  the  activity  which  goes  into  the  conduct  of 
such  exercises  and  the  evaluations  based  on  them.  A  great  deal  of 
important  preparatory  activity  takes  place  before  the  exercise,  and 
much  of  the  analysis  and  synthesis  for  the  final  evaluation  must  be 
done  afterwards.  The  CAEF  must  be  designed  with  this  whole  cycle  in 
mind,  and  not  focused  too  heavily  on  the  exercise  period  per  se. 

Because  of  the  ongoing  nature  of  the  CAEF  and  the  size  of  the 
permanent  party  staff  likely  to  be  assigned,  it  seems  reasonable  to 
assume  that  it  would  be  exercised  on  a  regular  and  routine  basis. 

That  is  to  say  that  the  scheduling  of  exercises  would  not  be  driven 
solely  by  evaluation  questions  but  would  also  be  influenced  by 
considerations  of  facility  utilization.  The  sequence  of  events 
associated  with  a  particular  exercise  would  probably  be  similar  to 
that  shown  in  the  exercise  cycle  in  Fig.  2. 

The  cycle  begins  several  months  before  the  exercise,  as  the 
evaluation  objectives  are  formalized.  These  exercise  objectives  would 
probably  be  negotiated  between  various  interested  parties  both 
internal  to  and  external  to  the  facility.  The  various  aspects  of 
command  and  control  systems  and  processes  which  might  be  evaluated  in 


a  particular  exercise  could  include,  for  example,  organizational  or 
procedural  alternatives,  new  or  modified  sensor  or  communications 
systems,  automated  aids  under  consideration,  or  doctrinal 
alternatives . 
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Fig.  2  —  The  exercise  cycle 


Development  of  the  exercise  plan  would  then  begin.  Depending  on 
the  nature  of  the  evaluation  objectives,  a  scenario  would  be  chosen  to 
set  the  stage  for  the  exercise.  Since  the  exercise  is  conducted  in  a 
free  play  mode,  there  is  no  detailed  scenario  to  be  followed  as  the 
exercise  progresses;  however,  the  overall  objectives  and  operations 
plans  for  both  friendly  and  enemy  forces  must  be  established  to 
provide  a  baseline  for  conducting  the  conflict  consistent  with  the 


evaluation  requirements.  This  might  be  a  new  scenario  or  an  update  of 
a  stock  scenario  already  available. 

Modifications  would  be  made  to  accommodate  whatever  variations  in 
equipment,  organization  or  procedure  might  be  required  for  the  current 
exercise.  New  systems  should  not  be  evaluated  using  old  environments 
and  procedures.  Players  must  learn  how  to  use  new  things  in  the  best 
way  before  they  can  be  fairly  evaluated,  and  this  experience  should  be 
acquired  during  this  pre-exercise  phase  so  the  evaluation  can  be  done 
on  a  smooth,  efficient  operation  of  a  modified  system.  Existing 
procedures,  subsystems  and  supporting  systems  should  be  adapted  to 
best  accommodate  test  system  configurations. 

Development  of  an  evaluation  and  documentation  plan  would  take 
place  concurrently  with  the  exercise  development.  Evaluation 
objectives  would  guide  the  definition  of  data  to  be  collected  and 
observations  to  be  made.  Some  of  this  data  could  be  picked  off 
automatically  by  the  computer  during  the  course  of  the  exercise,  while 
some  would  require  the  presence  of  knowledgeable  human  observers. 
Provisions  would  be  made  for  more  complete  documentation  than  that 
required  by  the  exercise  objectives,  in  order  to  support  the 
historical  database. 

Close  coordination  should  occur  between  exercise  planners  and 
evaluation  planners  throughout  the  development  cycle.  Exercise 
planners  would  probably  be  almost  entirely  facility  personnel,  while 
the  evaluation  planners  would  come  from  the  facility  as  well  as  from 
outside  agencies  involved  with  whatever  questions  were  the  focus  of 
evaluation. 

This  preparatory  activity  eventually  culminates  in  the  exercise 
itself,  and  during  the  exercise  period  most  activity  at  the  facility 
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would  be  focused  on  running  and  monitoring  the  exercise.  Appropriate 
data  would  be  gathered  both  on-line  and  by  human  observers.  This 
would  include  data  specific  to  that  exercise  as  well  as  general 
documentation  data  gathered  routinely.  The  human  observers  would  be 
alert  to  notice  things  which  were  important  to  the  play  of  the 
exercise  but  which  had  not  been  anticipated  in  the  evaluation  design. 

Following  the  exercise  the  evaluation  team  would  debrief 
significant  participants,  including  players,  key  support  team 
personnel,  and  observer/ evaluators .  Debriefings  would  include  a 
combination  of  items  specific  to  that  evaluation  plan,  items  required 
for  the  ongoing  history/comparison  database,  and  participants' 
serendipitous  observations .  They  might  take  the  form  of  one-on-one  or 
group  debriefings,  of  post-exercise  questionnaires  or  other  survey 
instruments,  or  of  combinations  of  these.  The  evaluation  team  would 
then  prepare  one  or  more  evaluation  reports  for  that  exercise  as  well 
as  the  summaries  and  database  updates  necessary  for 
historical/comparative  purposes. 

The  structure  of  the  exercise  itself  is  shown  in  Fig.  3.  The 
Command  and  Control  Components  (C2  Components)  being  exercised  are 
embedded  within  an  Exercise  Driver  which  simulates  the  war  and  the 
rest  of  the  world  to  the  C2  Components  and  receives  and  implements 
force  allocations  and  other  command  and  control  actions  from  it. 
Sitting  across  the  top  of  both  is  the  Evaluation  Unit .  collecting  data 
on  which  to  base  later  evaluation  as  well  as  providing  whatever 
guidance  to  the  course  of  the  exercise  is  necessary  to  ensure  that 
evaluation  objectives  are  adequately  met. 
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Each  of  these--the  C2  Components,  Exercise  Driver,  and  Evaluation 
Unit- -should  be  thought  of  as  integrated  man/machine  units  composed  of 
both  humans  and  computers  and  computer  programs.  In  terms  of  the 
distinctions  usually  associated  with  military  exercises,  the  C2 
Components  consist  of  the  "players"  and  the  systems  and  facilities 
they  employ,  such  as  communications  systems  and  facilities. 


Fig.  3  —  Computer  aided  evalution  exercise 


intelligence  databases,  and  weaponeering  computers.  The  Exercise 
Driver  encompasses  much  of  the  "control  team"  and  the  computer  support 
to  provide  the  kind  of  responsive  exercise  environment  necessary  for 
effective  evaluation.  The  Evaluation  Unit  includes  the  "evaluators" 
and  the  computer  subsystems  which  support  their  functions.  The 
Exercise  Driver  and  the  Evaluation  Unit  must  overlap  somewhat,  in  that 
the  Evaluation  Unit  must  maintain  sufficient  management  control  over 
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the  course  of  the  exercise  to  ensure  that  evaluation  objectives  are 
met. 

A  primary  issue  to  be  faced  in  the  design  of  the  CAEF  is  the 
relative  role  of  man  and  machine  in  the  conduct  and  control  of  the 
exercise.  At  one  extreme,  one  could  envision  a  fully  automated 
Exercise  Driver  and  Evaluation  Unit  in  which  all  necessary  human 
choices  and  decisions  were  made  prior  to  the  exercise  and  the  only 
role  for  human  beings  during  the  course  of  the  exercise  was  to  act  as 
intermediaries  between  the  computer  and  the  C2  Components  when  voice 
communication  was  required.  At  the  other  extreme,  one  can  imagine  an 
essentially  manual  exercise,  with  a  great  deal  of  automated 
bookkeeping  support  allowing  a  human  control  team  to  run  a  far  more 
flexible  and  responsive  exercise  than  is  now  generally  the  case. 

The  first  extreme  is  probably  not  feasible,  and  would  not  be 
desirable  even  if  it  were.  The  second  is  feasible,  but  falls  far 
short  of  the  the  kinds  of  capabilities  which  an  evaluation  exercise 
should  have.  The  proper  mix  of  man  and  machine  lies  somewhere  in  the 
middle,  with  the  machine  making  many  routine  decisions  and  providing 
more  than  bookeeping  support  for  others,  but  with  human  controllers 
retaining  ultimate  responsibility  for  the  management  and  direction  of 
the  exercise. 

The  evaluation  exercise  can  be  thought  of  as  consisting  of  two 
distinct  parts--the  exercise  per  se  as  a  simulated  conflict  involving 
only  the  interaction  of  the  C2  Components  and  the  Exercise  Driver,  and 
the  evaluation  of  the  role  of  command  and  control  in  that  conflict  by 
the  Evaluation  Unit.  The  overall  problem  of  designing  the  evaluation 
exercise  can  then  be  broken  down  into  distinct  simulation  and 
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evaluation  problems. 


Each  of  these  will  be  discussed  separately 


below. 
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III.  THE  EXERCISE  DRIVER 

The  Exercise  Driver  is  the  core  of  the  evaluation  exercise.  It 
must  be  able  to  provide  a  simulated  conflict  environment  which  is 
sufficiently  realistic  and  challenging  to  the  C2  Components  to 
encourage  realistic  performance,  and  which  is  sufficiently  responsive 
to  actions  taken  by  the  C2  Components  to  allow  that  performance  to 
affect  the  course  of  the  conflict  in  reasonable  ways. 

To  do  this,  the  Exercise  Driver  must  maintain  a  picture  of  the 
conflict  and  the  state  of  the  world  in  general  on  an  up-to-date  basis. 

It  must  accept  directions  from  the  C2  Components  for  force  allocations 
or  other  actions  under  their  control,  and  must  itself  take  appropriate 
actions  for  other  actors  in  the  conflict  process.  These  include  the 
enemy,  neutrals  and  allies,  higher  headquarters,  and  elements  of 
friendly  forces  not  represented  by  the  C2  Components.  It  must  update 
its  picture  of  the  state  of  the  world  appropriately  to  account  for 
these  actions.  It  must  also  feed  back  appropriate  information 
concerning  these  changes  to  the  C2  Components  in  the  same  formats  and 
with  the  same  distortions  and  time  delays  which  would  be  expected  in 
the  real  world. 

The  design  of  an  Exercise  Driver  to  perform  these  functions 
requires  a  number  of  choices  for  which  there  is  no  single  best 
solution.  Some  of  these  choices  revolve  around  such  issues  as  the 
relative  roles  of  man  and  machine,  the  degree  of  automation,  and  the 
appropriate  degree  of  modularity  in  the  design  of  the  CAEF  as  a  whole 
as  well  as  its  components.  More  will  be  said  about  these  issues  in 
what  follows,  to  set  the  stage  for  a  preliminary  design  from  which  to 

develop  a  functioning  CAEF.  _ _ 
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The  approach  to  the  design  itself  must  necessarily  be  different 
from  the  normal  research  and  analysis  approach  because  the  processing 
requirements,  manual  and  computer,  are  driven  primarily  by  on-line 
human  information  needs.  In  particular,  the  players  have  to  be 
serviced  with  information  which  is  in  both  form  and  substance  like 
that  they  would  receive  in  actual  practice.  Likewise,  the  staff  of 
the  Exercise  Driver  must  have  information  continually  to  perform  their 
tasks.  For  both  groups,  the  information  must  be  available  in  "live" 
time  rather  than  an  arbitrary  "simulation"  time.  Furthermore,  the 
live  play  aspect  of  this  approach  dictates  that  events  (such  as 
aircraft  takeoffs,  target  strikes,  ground  force  movements,  etc.),  be 
paced  to  live  time  as  well,  since  nearly  all  events  can  be  affected  by 
decisions  at  any  time.  And  also,  the  simulation  must  be  interruptible 
to  be  responsive  to  the  man-machine  interactions  which  dominate  this 
mode  of  operation. 

Functionally,  the  Exercise  Driver  can  be  thought  of  as  having  a 
structure  like  that  depicted  in  Fig.  4.  It  consists  of  three  modules 
--  the  Conflict  Module,  the  Interface  Module  and  the  Control  Module. 
The  Conflict  Module  simulates  and  keeps  track  of  the  conflict 
environment.  The  Interface  Module  provides  the  interfaces  between  the 
Exercise  Driver  and  the  C2  Components .  The  Control  Module  plays  the 
roles  of  other  participants  and  generally  oversees  the  functioning  of 


the  exercise. 


THE  CONFLICT  MODULE 


The  Conflict  Module  must  simulate  and  monitor  the  entire  conflict 
environment  (including  the  location  and  operational  status  of  the  C2 
Components  and  their  supporting  systems)  in  sufficient  detail  to  drive 
the  exercise  with  an  acceptable  degree  of  realism  and  challenge.  In 
addition  to  modeling  the  interactions  between  opposing  forces,  it  must 


Fig.  4  —  Exercise  conflict  structure 


keep  track  of  status  and  movement  of  friendly  and  enemy  force  elements 
and  the  status  and  actions  of  all  decision  making  participants  in  the 
conflict,  live  or  simulated,  including  the  C2  Components.  The 
Conflict  Module  maintains  the  primary  data  base  of  information  about 
the  conflict  environment. 
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The  scope  of  the  conflict  will  be  determined  by  the  evaluation 
requirements,  but  for  design  sizing  considerations  the  Conflict  Module 
should  be  capable  of  representing  "theater"  level  air/land  warfare. 

It  should  be  able  to  represent  two  corps  areas  in  detail  and 
additional  corps  on  an  aggregated  basis.  It  must  include  theater 
level  airfields  and  rear  area  targets  on  both  sides  and  be  capable  of 
representing  two-sided  tactical  air  operations  throughout  the  theater. 
It  should  be  capable  of  simulating  theater  intelligence  assets,  and  of 
supporting  the  simulation  of  non-theater  assets  such  as  national 
intelligence  systems.  That  is  to  say,  if  these  assets  are  not 
simulated  directly,  the  Conflict  Module  should  be  capable  of 
interacting  with  appropriate  external  simulations  and  ensuring  that 
data  bases  used  by  those  simulations  are  consistent  with  that 
maintained  by  the  Conflict  Module.  Logistics  and  logistic  support 
systems  should  also  be  represented. 

The  level  of  detail  required  of  the  Conflict  Module  will  vary  to 
meet  information  needs  of  the  players,  the  support  staff  and  the 
evaluation.  The  requirement  will  depend  not  only  on  operational 
considerations,  such  as  type  of  air  mission,  but  also  on  which  command 
and  control  activities  are  being  played  live  and  which  are  being 
represented  by  the  Control  Module.  In  many  cases  an  aggregated  or 
expected  value  calculation  will  suffice  for  determining  force 
engagement  outcomes  or  satisfying  evaluation  estimates,  but  such 
things  as  aircraft  attrition  and  bomb  damage  assessment  must  be 
reported  in  specifics  to  provide  the  players  realistic  reports.  Some 
form  of  hierarchical  structure  of  detail  in  handling  the  conflict 
environment  data  will  probably  be  required  for  the  simulation  to  be 


efficient . 
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The  pace  of  the  conflict  must  reasonably  reflect  the  capabilities 
of  both  sides.  This  is  of  particular  importance  in  the  ground 
conflict,  which  may  be  expected  to  proceed  in  a  more  "unattended" 
fashion  than  the  air  operations  since  the  latter  are  paced  by  the  C2 
Components  being  played.*  Events  must  occur  no  faster  than  live  time 
in  order  to  assure  that  the  multiplicity  of  independent,  parallel 
activities  taking  place  are  consistent.  Since  decisions  on  force 
application  can  be  made  at  any  time  by  either  the  players  or 
supporting  staff,  it  is  never  possible  to  predict  future  events  with 
certainty.  In  fact,  it  is  the  purpose  of  command  and  control  to 
manipulate  force  elements  in  response  to  perceived  actions,  events  and 
situations  as  they  are  recognized,  and  do  that  in  unpredictable  ways. 

Although  a  principal  function  of  the  Conflict  Module  is  to 
simulate  and  keep  track  of  the  conflict  environment  during  the  course 
of  an  exercise,  it  has  additional  "off  line"  functions  which  impact  on 
its  design  characteristics.  It  may  be  used  by  the  Control  Module 
during  the  exercise  for  "look  ahead"  to  assess  the  potential  impact  of 
Control  Module  actions  before  they  are  actually  taken,  and  may  be  used 
by  the  CAEF  between  manned  exercises  for  replays  and  variations  on 
past  exercises  run  for  analytical  purposes.  In  addition,  it  will  be 
used  during  exercises  to  collect  and  summarize  exercise  history  data 
for  later  use  by  the  evaluation  unit.  It  should  therefore  be  designed 
with  appropriate  "pick  off"  points  and  other  collection  capabilities. 


*This  is  not  to  rule  out  the  live  participation  of  ground  force 
decisionmakers  playing  as  an  adjunct  to  the  C2  Components.  That  would 
be  most  desirable,  and  perhaps  essential  to  some  evaluations,  and 
should  be  considered  for  inclusion  as  the  CAEF  is  developed. 
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Design  Considerations 

What,  then,  should  be  the  underlying  design  of  the  Conflict 
Module?  We  have  already  committed  to  a  man/machine  structure,  but  the 
issues  are  how  much  man,  how  much  machine,  and  how  should  the  two  be 
structured.  Resolution  of  those  issues  is  the  heart  of  the  research 
problem  and  no  specific  conclusions  are  yet  available,  but  some  key 
considerations  are  worthy  of  mention.  Foremost  is  the  question  of  the 
degree  of  modularity,  particularly  in  the  computerized  portions. 

At  one  end  of  the  spectrum  is  a  very  large,  self-contained  combat 
model  which  essentially  does  everything  internally,  including 
interpreting  the  air  tasking  order  and  other  action  orders, 
determining  the  effect  of  the  associated  military  actions,  updating 
the  databases,  providing  output  in  appropriate  form  and  interacting 
with  the  supporting  staff.  At  the  other  end  is  a  collection  of 
relatively  small,  distinct  function  modules  (man,  machine  or  both) 
which  are  interconnected  to  perform  the  many  necessary  operations 
required  of  the  Conflict  Module. 

Design  around  a  single  large  combat  model  as  a  core  is  attractive 
on  the  surface  because  it  appears  to  provide  a  possibility  of 
exploiting  some  completed  work,  since  research  and  analysis  combat 
models  already  exist  which  provide  many  of  the  required  functions.  In 
the  best  case,  a  saving  of  time  and  money  to  implement  a  first-cut 
system  with  extensive  processing  capability  could  be  realized. 
Unfortunately,  such  savings  would  depend  on  being  able  to  use  a  model 
pretty  much  as  it  stands.  As  noted  above,  however,  the  necessity  to 
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satisfy  human  information  needs  dominates  this  processing  task,  and 
any  existing  model  would  have  to  be  adapted  to  that  concept. 
Adaptations  would  likely  include  man/machine  interaction  and 
generation  of  fine-grained  force  element  accounting  and  operation  in 
live  time  as  a  minimum.  So  the  relative  efficiency  of  adapting  an 
existing  model  versus  building  a  new  model  (perhaps  from  pieces  of 
existing  ones)  is  an  open  question  for  detailed  consideration  if  a 
single  model  design  were  to  be  selected. 

Another  factor  to  be  considered  is  the  facility  of  the  model  for 
modification.  It  can  be  expected  that  each  evaluation  exercise  will 
have  special  requirements  not  included  in  the  Conflict  Module, 
regardless  of  the  design  or  model  being  used;  hence,  it  would  be 
expected  that  the  model  would  have  to  be  changed.  Changing  large 
models  is  a  nontrivial  task  in  most  cases.  So,  either  a  major  effort 
will  need  to  be  made  or  the  model  itself  will  have  a  tendency  to 
inhibit  the  flexibility  of  the  Conflict  Module  or,  worse,  overly 
influence  the  design  of  the  evaluation  exercise,  possibly  to  the  point 
of  restricting  evaluation  goals.  Certainly,  some  of  this  will  occur 
under  any  design,  but  exercise  designs  that  encourage  it  should  be 
avoided. 

Modular  Design 

A  more  modular  design  would  have  several  advantages.  A  modular 
design  lends  itself  readily  to  evolutionary  development  in  a 
computer-aided  exercise  environment.  The  modules  would  be  man/machine 
systems,  with  greater  or  lesser  degrees  of  automation.  (A  manual 
exercise  environment  can  be  thought  of  as  a  modular  design  in  which 
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all  the  modules  are  fully  manual.)  As  a  basis  for  a  carefully 
coordinated,  systematic  evolution,  an  overall,  comprehensive  modular 
design  of  the  Conflict  Module  would  have  to  be  developed  first.  At 
least  initially,  depending  on  the  time  and  resources  available  for 
implementation,  many  of  the  functions  could  be  purely  manual  with 
selected  modules  automated  fully  and  others  to  some  extent.  The 
Conflict  Module  would  then  be  evolved  by  upgrading  modules  in  line 
with  the  overall  design  to  maintain  coherence  and  interoperability 
among  modules.  Lessons  learned  in  implementing  each  new  module  would 
be  applied  to  upgrading  and  installing  the  next  modules.  Development 
of  the  Conflict  Module  in  this  fashion  would  permit  pacing  consistent 
with  technological  advancements  and  the  availability  of  funds, 
facilities,  equipment  and  personnel.* 

A  modular  design  would  also  provide  flexibility  in  tailoring  the 
exercise  environment  to  particular  evaluation  needs  by  having  the 
computer  aids  in  small  hardware/software  packages  which,  generally, 
are  easier  to  modify  than  large  ones.  Also,  different  versions  of  the 
modules,  tailored  for  specific  applications,  could  be  maintained  and 
used  selectively  as  needed.  Updating  and  modernization  would  be 
facilitated  since  individual  modules  could  be  revised  one  at  a  time. 
And  there  is  the  potential  that  integrating  automated  command  and 
control  systems  into  the  CAEF  would  be  relatively  straightforward, 
since  the  modules  would  be  designed  to  accept  and  work  with  a  variety 


*To  some  extent  this  process  is  occurring  now  anyway,  as  automat¬ 
ed  aids  of  various  kinds  are  introduced  into  the  exercise  environment. 
But  most  of  these  introductions  are  ad  hoc  and  uncoordinated,  while 
under  the  CAEF  concept  the  development  would  be  design-based,  sys¬ 
tematic  and  explicit. 
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of  interfaces.  Modular  design  might  be  significantly  facilitated  by 
the  selective  use  of  newer  programming  technologies  such  as  rule-based 
programming  in  place  of  more  traditional  programming  techniques  where 
they  apply. 

One  way  to  structure  a  modular  design  would  be  to  organize  it 
around  the  functions  which  the  Conflict  Module  needs  to  perform  in 
running  an  exercise.  One  such  list  might  be: 

o  Database  modules  to  establish  and  maintain  the  primary 
databases  and  deal  with  different  data  aspects  such  as 
own  force  status,  enemy  force  status,  missions  in 
progress,  etc.  They  retain  sole  authority  to  update  or 
otherwise  change  the  databases  and  have  responsibility 
to  provide  all  information  from  the  databases  to  all 
users . 

o  Activity  modules  to  determine  (simulate,  compute, 

estimate,  etc.)  the  effects  of  various  types  of  activity 
such  as  force  engagements,  mobility,  and  attrition; 
provision  of  logistics;  and  reconnaissance,  surveillance 
and  other  intelligence  gathering  activities. 

o  Management  modules  to  interleave  the  database  and 

activity  modules  as  appropriate  to  simulate  the  results 
of  the  activities  on  the  conflict  environment,  cause 
database  updates  and  generate  output  as  appropriate. 

Within  this  basic  framework,  a  number  of  different  design 
configurations  using  differing  amounts  of  human  and  machine  resources 
would  be  possible.  And  even  a  minimal  design  consisting  of  only  a  few 


automated  database  modules  would  provide  a  significant  increase  in 
capability  over  a  manual  exercise  facility. 

Finally,  it  must  be  noted  that  most  likely  the  best  and  probably 
most  feasible  design  will  lie  somewhere  between  a  monolithic  model  and 
an  exhaustive  modularization.  There  will  always  be  a  large  degree  of 
modularity,  but  perhaps  with  some  rather  large  pieces. 


THE  INTERFACE  MODULE 

In  order  to  successfully  evaluate  a  command  and  control  system 
using  the  CAEF  approach,  the  C2  Components  must  operate  in  essentially 
the  same  environment  with  the  same  capabilities  and  constraints  as 
would  be  the  case  in  war.  To  the  maximum  extent  possible,  the  players 
should  perceive  and  interact  with  the  conflict  being  simulated  in  the 
same  way  that  they  would  perceive  and  interact  with  a  real  conflict  in 
the  field.  This  places  two  requirements  on  the  Exercise  Driver.  One 
is  that  the  simulated  conflict  environment  be  realistic.  The  other  is 
that  information  about  the  conflict  should  enter  the  C2  Components  the 
same  as  it  would  in  actual  conflict  and  information  sent  out  by  the  C2 
Components  should  go  out  in  the  same  way,  in  the  same  form  and  via  the 
same  equipment  as  in  a  real  situation. 

It  is  the  job  of  the  Interface  Module  to  see  that  the 
interactions  between  the  C2  Components  and  the  conflict  being 
simulated  meet  these  criteria.  As  indicated  in  the  design  shown  in 
Fig.  4,  all  communications  between  the  C2  Components  and  the  Exercise 
Driver  are  processed  through  the  Interface  Module.  Furthermore,  to 
enable  simulated  degradation  of  communications,  most  communications 
strictly  between  individual  C2  Components  ultimately  should  also  be 
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rout  ed  through  the  Interface  Module.  The  Interface  Module  directs  C2 
Components'  outputs  to  either  the  Conflict  Module  or  Control  Module 
for  appropriate  action.  Going  the  other  way,  the  Interface  Module 
assures  that  inputs  to  the  C2  Components  are  in  the  proper  form  and 
transmits  them  by  correct  media  with  accuracy  consistent  with  the 
current  simulated  state  of  the  communications  systems  being  used. 

Inputs  to  the  C2  Components  are  of  four  general  types:  (1)  data 
about  the  enemy,  (2)  data  about  friendly  forces,  both  ground  and  air 
but  particularly  details  of  own  air  resources,  (3)  guidance,  direction 
and  planning  factors  from  other  command  and  control  activities  and  (4) 
intercommunications  among  the  C2  Components  concerning  their 
functional  and  procedural  needs.  Additionally,  there  are 
miscellaneous  inputs  such  as  administrative  information,  weather  data, 
terrain  conditions,  sea  states  and  other  items  not  falling  into  the 
four  given  above  but  used  in  the  command  and  control  processes.  The 
majority  of  these  inputs  are  in  some  form  of  hardcopy  report  whose 
content  and  format  are  specified  in  military  manuals. 

Outputs  from  the  C2  Components  include  the  air  tasking  order  for 
the  following  day’s  operations;  pre-mission  modifications  to  the  air 
tasking  order;  real  time  diversions  of  aircraft  to  other  missions, 
targets  or  recovery  bases;  and  other  operations  orders  such  as  force 
deployments.  Depending  on  the  level  and  components  being  played, 
outputs  may  also  include  direct  radio  communications  with  (simulated) 
aircraft.  Also,  coordination  actions  and  discussions  with  other 
forces'  command  and  control  activities,  intercommunications  among  the 
C2  Components  and  requests  for  additional/updated  input  information 
from  various  sources  generate  outgoing  communication  requirements. 
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Communications  Interfaces 

Currently,  this  information  is  transmitted  to,  from  and  within 
the  command  and  control  system  primarily  by  teletype  (TWX),  telephone 
(voice),  and  in  some  cases  radio  (also  voice)  means.  Utilization  of 
computer -to -computer  communications  is  increasingly  being  developed 
and  can  be  expected  to  carry  a  large  portion  of  the  traffic, 
eventually.  The  form  of  the  information  varies  from  rigid  formatted 
data  entries  and/or  free  flowing  text  in  standard  reports  to 
conversational  style  in  either  voice  or  TWX  mode. 

The  Interface  Module  must  be  able  to  perform  every  combination  of 
communications  mode  and  form  appropriate  for  any  given  situation. 
Interfaces  can  be  characterized  as  automated,  manual  or  combined 
depending  on  the  degree  to  which  people  must  be  involved  in  either 
preparing  or  transmitting  the  information.  An  example  of  an  automated 
input  interface  would  be  one  in  which  the  information  was  completely 
determined  by  computer,  correctly  formatted  by  computer  and  directly 
transmitted  by  computer  from  the  Exercise  Driver  to  a  C2  Component  via 
teletype  or  computer -to -computer  means .  Some  of  the  highly  formatted 
intelligence  and  force  status  reports  are  candidates.  A  manual  input 
interface  would  be  one  in  which  the  entire  process  of  data  extraction, 
formatting  and  transmission  was  performed  by  people.  On-line 
responses  to  telephonic  queries  and  open  discussions  with  Control 
Module  personnel  acting  as  other  command  elements  would  be  examples. 

At  least  in  the  initial  phases  of  CAEF  development,  most  input 
interfaces  would  fall  into  the  combined  category  wherein  the  computer 
performs  data  extraction  from  the  data  bases  via  a  man/machine 
interaction  but  that  information  must  be  interpreted  and  formatted  by 
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a  person.  Textual  and  conversational  inputs  would  generally  be  in 
this  category,  although  computer  generation  of  textual  materials  is  a 
feasible  capability  which  can,  and  should,  be  developed.  Voice 
transmission  would  always  require  a  direct  human  interface;  however, 
in  many  cases,  the  determination  of  both  the  information  and  the  need 
to  send  it,  as  well  as  the  formatting,  could  be  done  by  computer. 
Verbal  mission  reports  and  high  priority  critical  damage  (from  enemy 
action)  reports  are  examples  where  the  condition  could  be  recognized 
by  the  computer  as  the  event  is  processed  and  a  report  automatically 
generated  and  provided  to  appropriate  personnel  for  voice 
transmission. 

Until  more  automated  systems  are  incorporated  into  the  tactical 
air  command  and  control  system,  the  air  tasking  order  and  changes 
thereto  will  probably  be  the  only  candidates  for  automated  output 
interfaces  (those  that  require  no  interaction  by  the  Exercise  Driver 
personnel  to  execute  the  indicated  actions)  from  the  C2  Components. 
Most  other  teletype  messages  will  be  received  from  the  C2  Components 
over  digital  circuits,  but  are  likely  to  require  a  person  to  read  and 
interpret  them.  It  should  be  recognized  that  regardless  of  the  form 
and  media  actually  used  at  the  receiving  end  of  an  output  to  the  real 
world,  if  the  Exercise  Driver  is  the  receiver  of  the  output  during  the 
exercise,  the  form  and  media  can  be  different.  Eventually  the 
preferred  method  of  receiving  things  in  the  Exercise  Driver  will  be 
directly  into  a  computer  for  display  on  a  terminal  at  a  work  station, 
whether  or  not  it  is  processed  automatically.  Paper  shuffling  needs 
to  be  minimized  during  high  intensity  operations. 

One  of  the  tasks  in  designing  the  Interface  Module  is  to 


determine  what  form,  substance  and  communications  media  are 


appropriate  for  all  data  passed  between  the  C2  Components  and  the 
Exercise  Driver  and  develop  the  necessary  interfaces •-automated, 
manual  or  combined.  The  elimination  of  most  purely  manual  interfaces 
will  be  one  of  the  most  important  advances  realized  in  the 
construction  of  a  CAEF.  The  evolution  of  the  Interface  Module  should 
emphasize  reducing  dependency  on  manual  activity  wherever  possible  to 
expedite  and  increase  the  accuracy  of  information  transfers.  However, 
the  degree  of  automation  possible  for  the  Interface  Modules  would  vary 
somewhat  with  the  nature  of  the  C2  system  being  exercised.  Exercises 
simulating  the  current  system  with  its  heavy  dependence  on  voice 
communications  would  require  far  more  manual  interface  than  exercises 
simulating  systems  more  heavily  dependent  on  digital  communications. 

The  Interface  Module  would  also  be  responsible  for  simulating  the 
capabilities  of  the  communications  systems  used  to  support  the  command 
and  control  system.  It  must  introduce  such  noise,  interference  and 
interruption  as  would  be  experienced  in  a  real  communications  system 
under  the  simulated  conflict.  This  must  be  done  in  both  directions, 
into  and  out  of  the  C2  Components.  Communications  sent  in  must 
include  transmission  errors  such  as  garbling  and  omissions,  and  those 
sent  out  must  also  be  degraded  to  represent  real  world  problems  in 
working  with  imperfect  communications.4' 


*This  type  of  degradation  should  not  be  confused  with  imperfect 
Information  about  the  status  and  posture  of  friendly  and  enemy  forces 
gathered  by  less  than  perfect  collection  systems.  The  accuracy  and 
completeness  of  information  provided  to  the  C2  Components  will  depend 
on  the  the  quality  of  collection  and  reporting  systems  and  how  they 
are  applied.  Determination  of  what  information  to  make  available  for 
transmission  to  the  C2  Components  is  a  function  of  the  Conflict  and 
Control  Modules. 


-29- 


Design  Considerations 

The  advantages  of  modularity  discussed  earlier  apply  to  the 
Interface  Module  as  well.  The  Interface  Module  requirements  suggest 
modularization  along  the  following  functional  lines. 

o  Communicat ions  module  to  perform  the  actual  transmission 
of  information  to  and  from  the  C2  Components.  It  would 
represent  the  capabilities  of  the  communications  systems 
being  played  in  the  exercise  and  reflect  degradation  to 
those  systems  as  appropriate  from  enemy  actions  or  other 
causes . 

o  Processing  module  to  interact  with  the  Control  and 

Conflict  Modules  to  acquire  information  for  transmission 
to  the  C2  Components.  Processing  would  be  initiated  both 
on  demand  from  requests  for  information  and  on  a  normal 
reporting  schedule  driven  by  either  time  or  events. 

Again,  the  modularity  approach  would  support  an  evolutionary 
development  of  the  CAEF  by  permitting  small  automated  modules  to  be 
activated  and  installed  as  resources  allow,  while  maintaining  an 
operational  capability  with  decreasing  manual  effort. 
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THE  CONTROL  MODULE 


The  Control  Module  has  responsibility  for  online  control  of  the 
exercise.  The  exercise  plan  prepared  in  the  pre-exercise  period 
provides  the  basis  for  conducting  the  exercise  but  the  free  play 
approach  essential  to  the  CAEF  concept  will  demand  continual  real  time 
adjustments  to  achieve  the  established  goals.  The  Control  Module  will 
continuously  monitor  actions  taken  by  the  C2  Components  and  events 
taking  place  in  the  conflict  environment  in  order  to  provide  the 
necessary  direction  to  the  Conflict  and  Interface  Modules  to  keep  the 
exercise  on  track. 

Although  it  is  the  responsibility  of  the  Evaluation  Unit  to 
establish  evaluation  goals  and  ensure  the  accomplishment  of  those 
goals  during  the  exercise,  the  Control  Module  should  have  sole 
authority  and  responsibility  for  the  conduct  of  the  exercise.  Close 
coordination  between  the  Control  Module  and  the  Evaluation  Unit  is 
imperative,  and  either  group  can  identify  actual  or  potential 
discrepancies  which  would  endanger  successful  evaluation.  However, 
once  the  new  exercise  requirement  has  been  established,  it  is  up  to 
the  Control  Module  to  determine  what  adjustments  are  necessary  and 
make  them. 

The  Control  Module  also  performs  the  roles  of  command  and  control 
actors  not  represented  by  the  C2  Components.  These  might  include  Air 
Force  and  joint  authorities  higher  in  the  command  chain,  any  lateral 
or  lower  Air  Force  command  and  control  elements  not  being  played, 
friendly  ground  force  commanders  and  enemy  decisionmakers.  The 
actions  taken  in  the  name  of  these  actors  must  be  representative  of 
their  expected  actions  in  the  air/land  conflict  being  simulated.  But 


furthermore,  the  actions  must  recognize  the  exercise  objectives  and 
the  overall  plans,  strategies  and  goals  of  the  conflict  set  forth  in 
the  exercise  plan  and  maintain  consistency  with  them.  So,  actions 
must  be  carefully  chosen  based  bn  both  the  current  conflict  situation 
and  what  effect  those  actions  are  likely  to  have  in  the  future. 

Information  available  to  the  surrogate  decisionmakers  could 
either  be  absolute  "ground  truth"  from  the  Conflict  Module  or  degraded 
as  it  is  to  the  C2  Components;  however,  the  actions  of  friendly 
authorities  should  reflect  no  more  information  than  would  reasonably 
be  available  to  the  entities  being  simulated.  This  is  especially 
important  during  telephonic  conversations  between  players  and 
simulated  higher  headquarters.  Actions  by  enemy  decisionmakers  should 
be  in  line  with  estimates  of  enemy  capability  to  collect  and 
promulgate  information.  Even  adjustments  to  the  course  of  the 
exercise  to  pursue  evaluation  goals  must  be  accomplished  through 
realistic  events  and  actions. 

Design  Cons ider at ions 

Modularity  in  designing  the  Control  Module  would  follow 
functional  lines  suggested  above. 

o  Monitor  module  to  provide  the  capability  to 
selectively  monitor  exercise  actions  and  events 
and  situational  and  status  factors  pertaining  to 
the  simulated  conflict. 
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o  Look-ahead  module  to  provide  the  capability  to 
forecast  future  conflict  situations  under  assumed 
courses  of  action  or  event  occurrences, 
o  Friendly  decisionmaking  module  to  provide  the 
capability  to  make  friendly  command  and  control 
decisions  and  effectively  interact  by  voice  with 
the  C2  Components. 

o  Enemy  decisionmaking  module  to  provide  the 

capability  to  make  command  and  control  decisions 
for  the  enemy  air  and  land  forces. 

As  with  the  Conflict  and  Interface  Modules,  such  a  modular  design 
would  aid  the  evolutionary  development  of  the  CAEF. 
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IV.  m  COMMAND  AND  CONTROL  COMPONENTS 

The  purpose  of  a  computer-aided  evaluation  exercise  is  to  provide 
a  simulated  environment  within  which  to  exercise  and  evaluate  the 
performance  of  the  tactical  air  command  and  control  elements  contained 
in  the  C2  Components.  In  any  given  exercise  these  components  consist 
of  the  people,  organizational  structure  and  supporting  systems 
corresponding  to  the  elements  being  played. 

During  the  early  stages  of  development  of  the  CAEF,  the  focus  may 
be  on  playing  a  Tactical  Air  Control  Center  (TACC)  or  its  NATO 
counterparts.  As  development  proceeds,  capabilities  to  play 
additional  elements,  possibly  including  Army,  would  be  added.  The 
particular  elements  which  play  as  part  of  the  C2  Components  will,  of 
course,  vary  with  the  exercise  and  exercise  objectives,  and  relevant 
portions  not  being  played  as  part  of  the  C2  Components  will  be 
simulated  by  the  Exercise  Driver.  For  example,  when  the  questions 
being  investigated  revolve  primarily  around  the  management  of  tactical 
air  resources  for  interdiction  or  offensive  counter  air,  the  C2 
Components  might  consist  of  nothing  but  a  Tactical  Air  Control  Center 
(TACC)  or  its  NATO  counterparts,  with  all  other  elements  simulated 
within  the  Exercise  Driver.  On  the  other  hand,  when  questions  related 
to  offensive  air  support  operations  are  being  investigated,  it  would 
be  important  to  include  such  elements  as  the  Direct  Air  Support 
Centers  (DASC)  in  the  C2  Components  as  well.  And  for  some  purposes, 
it  might  be  desirable  to  play  even  more  elements,  such  as  a  Control 
and  Reporting  Center  (CRC)  and  Ving  Operations  Centers  (WOC). 


Ve  are  conceptualizing  the  CAEF  as  an  Air  Force  facility  intended 
to  exercise  tactical  air  command  and  control,  with  other  corasand  and 
control  elements  present  only  as  part  of  the  Exercise  Driver.  There 
is  no  reason  in  principle,  however,  why  Army  C2  elements  as  well  could 
not  be  accommodated  within  the  C2  Components.  This  might  be  done  by 
expanding  the  CAEF  itself  to  include  Army  elements,  or  by  establishing 
appropriate  interfaces  between  the  CAEF  and  a  comparable  Army 
facility. 

The  command  and  control  system  elements  played  in  the  C2 
Components  and  those  simulated  by  the  Exercise  Driver  must  be  treated 
differently  in  that  a  realistic  and  challenging  conflict  environment 
must  be  provided  for  the  C2  Components,  while  the  Exercise  Driver  is 
the  thing  which  provides  that  environment.  Thus  elements  simulated  by 
the  Exercise  Driver  may  be  very  different  in  form  and  function  from 
their  real  life  counterparts — one  man  may  operate  an  entire  simulated 
DASC  or  several  simulated  WOCs,  for  example. 

The  elements  in  the  C2  Components,  however,  should  be  as  close  as 
possible  in  form  and  function  to  their  real  life  counterparts .  They 
should  have  the  same  manning,  the  same  equipment,  the  same  physical 
layout,  and  should  follow  the  .  w  procedures.  When  variations  from 
existing  systems  or  procedures  are  exercised,  they  should  conform  as 
closely  as  possible  to  what  would  be  expected  to  be  fielded. 

This  means,  in  particular,  that  systems  or  procedures  which  would 
be  employed  in  the  field  by  personnel  experienced  in  their  use  should 
not  be  employed  in  evaluation  exercises  by  players  who  have  not 
previously  used  them.  This  suggests  that  the  pre-exercise  preparation 
for  exercises  in  which  new  systems  or  procedures  are  to  be  evaluated 


might  include  a  series  of  training  and  familiarization  runs  to  bring 
the  C2  Components  personnel  up  to  an  appropriate  experience  level  with 
those  new  systems  or  procedures .  _  This  type  of  pre-exercise  training 
should  be  much  easier  to  design  and  conduct  with  the  CAEF  than  it  is 
with  existing  manual  exercise  facilities. 
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V.  COMMAND  AND  CONTROL  EVALUATION 

The  purpose  of  the  CAEF  is  to  conduct  exercises  as  a  foundation 
for  the  understanding  and  evaluation  of  command  and  control  processes 
and  systems.  However,  the  evaluation  of  the  contribution  of  command 
and  control  to  the  combat  effectiveness  of  military  forces  is  an  ill- 
defined  and  poorly  understood  area,  in  need  of  considerable  conceptual 
and  intellectual  development.  Sufficient  development  of  concepts  and 
techniques  needs  to  be  completed  to  provide  a  basis  on  which  to 
establish  an  initial  capability,  but  the  CAEF,  and  in  particular  the 
Evaluation  Unit,  should  be  thought  of  in  part  as  a  testbed  within 
which  further  development  can  take  place.  The  evaluation  approach, 
then,  is  not  something  which  can  be  laid  out  fully  before  the  CAEF  is 
built.  Rather,  it  requires  the  articulation  of  some  initial  exercise 
evaluation  concepts  and  a  design  permitting,  and  specifying  the 
responsibility  for,  self-evolution. 

THE  NATURE  OF  TACTICAL  COMMAND  AND  CONTROL  EVALUATION 

Although  any  particular  evaluation  that  is  conducted  will 
necessarily  be  tailored  to  respond  to  the  specific  context  and  purpose 
of  the  questions  or  issues  which  generate  the  evaluation  requirement, 
there  are  some  basic,  general  concepts  and  contexts  which  provide  the 
fabric  on  which  this  tailoring  is  done. 

There  are  three  broad  areas  of  interest  from  which  an  evaluation 
requirement  would  likely  be  generated.  One  is  the  internal  operation 
of  the  command  and  control  system,  focussing  on  the  efficiency  of  the 
system  in  executing  its  assigned  functions.  A  second  is  the 
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operational  arena,  focussing  on  the  ability  of  a  fielded  command  and 
control  system  to  support  tactical  air  operations  requirements.  And 
the  third  is  the  management  decision  process .  focussing  on  the 
selection  and  acquisition  of  major  alternatives  to  achieving  command 
and  control  functions.  Common  to  successful  evaluation  within  these 
three  interests  is  the  need  to  relate  command  and  control  capabilities 
to  the  effective  employment  of  the  tactical  air  resources. 

Herein  lies  the  major  conceptual  and  practical  difficulty  in  the 
evaluation  of  command  and  control.  A  successful  approach  to 
evaluation  and  the  definition  of  evaluation  criteria  depends  on  the 
existence  of  well-defined  relationships  between  the  command  and 
control  elements  characterizing  the  systems  being  evaluated  and  the 
outcomes  of  actions  in  which  those  systems  are  employed.  But  such 
relationships  are  usually  not  obvious.  In  some  cases  they  may  not 
exist,  even  in  principle,  because  of  the  extremely  complex  interaction 
of  men,  procedures,  and  equipment  and  the  fact  that  the  effectiveness 
of  that  interaction  in  a  particular  conflict  situation  may  be 
determined  by  unique  idiosyncratic  characteristics  of  that  situation 
to  a  much  greater  extent  than  for  most  other  military  systems.  The 
kinds  of  system  characteristics  which  can  be  laid  down  most  clearly  in 
a  set  of  specifications  may  affect  the  outcome  only  indirectly, 
through  the  support  they  provide  to  the  more  elusive  human  decision 
processes  directing  the  battle. 

One  way  of  conceptualizing  this  is  to  think  of  the  command  and 
control  system  as  "permissive,”  in  the  sense  that  it  allows  the  human 
decisionmakers  directing  the  battle  latitude  in  what  they  can  do  with 
the  forces  available.  Changes  in  command  and  control  capabilities 


‘39 


which  broaden  that  latitude  in  previously  constrained  directions  will 
allow  a  difference  in  outcomes,  but  whether  that  difference  is 
realized  or  not  will  depend  on  how  that  latitude  is  utilized.  (This 
is  also  dependent  on  whether  or  not  the  particular  conflict  scenario 
is  one  which  calls  for  latitude  in  that  particular  direction.  The 
relationship  between  command  and  control  capabilities  and  conflict 
outcomes  may  be  much  more  sensitive  to  variations  in  scenario  and 
tactics  than  is  the  case  for,  say,  weapons  system  performance 
parameters . ) 

This  does  not  imply  that  command  and  control  cannot  be  evaluated, 
nor  that  that  evaluation  cannot  be  careful  and  systematic  with  a  large 
quantitative  component.  But  it  does  imply  that  the  evaluation  of 
command  and  control  must  necessarily  have  a  large  subjective 
component,  and  is  likely  to  be  better  done  and  more  meaningful  in  the 
long  run  if  that  subjective  component  is  acknowledged  and  dealt  with 
carefully  than  if  it  is  not. 

Relating  Command  and  Control  to  Outcomes 

Tactical  air  command  and  control  may  be  seen  as  constituted  of 
Elements  of  doctrine,  organizational  structure,  procedures,  personnel, 
facilities,  equipment  and  communications  to  provide  authorities  at  all 
command  levels  the  capability  to  perform  the  Functions  of  planning, 
directing  and  controlling  necessary  to  accomplish  their  Purpose , 
meeting  mission  objectives  through  the  performance  of  tactical  air 
operations.  This  representation  is  shown  in  Fig.  5. 


Some  important  observations  can  be  made  from  this  representation. 
Only  the  Elements  have  well-defined,  measurable  attributes --quantity. 
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performance  factors,  physical  characteristics,  etc. --which,  taken 
together,  form  the  description  of  a  command  and  control  system.  Only 
the  Elements  can  be  added  to,  manipulated,  and  modified  to  produce 
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Fig.  5  —  Tactical  air  command  and  control 


variations  in  the  capabilities  of  command  and  control.  However,  the 
crucial  output  of  command  and  control  is  its  contribution  to  mission 
accomplishment,  and  the  impact  of  the  Elements  on  that  output  is  not 
direct,  but  indirect  through  the  Functions.  Performance  of  the 
Functions  is  dependent  not  only  on  the  existence  but  also  on  the 
utilization  of  the  Elements,  on  such  factors  as  how  procedures  are 
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followed  and  what  information  is  communicated.  But  even  more, 
performance  of  the  Functions  depends  heavily  on  human  judgment 
processes.  Planning,  directing  and  controlling  make  use  of  the 
Elements  and  what  they  provide,  but  plans,  directions  and  control 
actions  are  products  of  excisions  by  people. 

According  to  this  representation  the  Elements  are  the  basis  of 
command  and  control  capability  and  are  the  only  controllable  part. 
Changes  can  be  made  to  Elements,  and  decisions  that  are  made  about 
alternative  command  and  control  processes  or  systems  are  usually 
decisions  about  the  characteristics  of  the  Elements  making  up  that 
process  or  system.  On  the  other  hand,  relevant  evaluation  criteria 
for  command  and  control  must  focus  on  its  effect  on  the  employment  of 
the  tactical  air  resources,  and  the  accomplishment  of  the  tactical  air 
mission.  So  the  key  is  to  relate  the  Elements  to  mission 
accomplishment.  But  the  relationship  is  indirect  through  the 
Functions,  which  depend  on  the  interaction  of  the  Elements  and  human 
decisionmaking.  The  Elements  support  both  the  decisionmaking  process 
and  the  implementation  of  decisions  made  by  that  process.  Any 
evaluation  method  must  handle  this  indirect  relationship  to  determine 
cause  and  effect  between  Elements  and  measures  of  mission 
accomplishment . 

Mission  accomplishment  can  be  considered  at  two  levels--tactical 
air  operations  and  major  military  actions.  Tactical  air  operations 
are  the  basic  missions  flown  by  the  Air  Force  and  are  categorized  by 
general  objectives  and,  further,  by  target  and  force  management 
characteristics.  Close  Air  Support,  Interdiction  and  Air  Defense  are 
examples.  Measures  of  mission  accomplishment  at  this  level  focus  on 


the  ability  to  conduct  the  air  missions  themselves,  individually  or  in 
the  aggregate,  rather  than  on  the  effect  of  those  missions  on  combat 
outcome  or  the  course  of  the  conflict. 

By  a  major  military  action  we  mean  a  well-defined  and  bounded  air 
or  joint  air/land  action  requiring  deliberate  planning  and  execution 
of  persistent  tactical  air  operations  to  accomplish  particular 
objectives  considered  to  be  of  significant  military  value.  Included 
would  be  land  battles;  air  interdiction  campaigns  with  specific  goals 
(such  as  isolation  of  battle  zones);  counter  air  campaigns;  and 
aspects  of  a  land  battle  considered  crucial  by  a  ground  commander  and 
designated  by  him  to  be  primarily  an  air  support  responsibility  (such 
as  neutralization  of  all  long-range  artillery,  a  quota  of  armored 
vehicles  or  locally  critical  lines  of  communication). 

At  this  level  the  focus  is  on  the  course  of  the  conflict  and  the 
combat  outcome,  and  the  term  "mission  accomplishment"  has  the  larger 
meaning  of  goals  or  objectives  in  the  overall  combat  scene.  Measures 
of  mission  accomplishment  in  the  sense  of  achieving  these  broader 
objectives  are  concerned  with  what  overall  effect  tactical  air 
operations  might  have.  In  the  light  of  this  discussion,  then.  Fig.  5 
might  be  modified  as  shown  in  Fig.  6. 

Command  and  control  plays  a  role  in  the  conduct  of  tactical  air 
operations  in  many  ways,  including  determining  which  will  be 
conducted,  when,  where,  and,  to  a  great  degree,  how,  and  providing 
direct  inflight  support.  Its  role  in  affecting  major  military  actions 
includes  not  only  the  conduct  of  tactical  air  operations  but  also  the 
decisions  on  what  must  be  done  to  achieve  the  desired  effect  with  the 
available  resources.  Furthermore,  command  and  control  performs  the 


operational  management  of  the  tactical  forces,  and  how  it  performs 
that  task  will  influence  what  resources  are  available  to  accomplish 
missions. 


Elements 


Purpose  Functions 


Fia.  6  —  Tactical  air  command  and  control  related  to  outcomes 
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So,  evaluation  criteria  reflecting  effectiveness  of  force 
employment  can  encompass  measures  of  how  well  the  force  is 
operationally  managed  (resource  status)  as  well  as  of  the 
effectiveness  of  individual  air  operations  per  se.  Measures  of 
effects  on  major  military  actions  will  need  to  be  based  on  a  composite 
of  tactical  air  operation  effectiveness  and  a  comprehension  of  how 
that  effectiveness  impacts  on  the  outcomes  of  those  major  actions. 

For  completeness,  observe  that  resource  status  can  influence  tactical 
air  operation  effectiveness,  so  measures  linking  these  two  may  also 
provide  measurable  links  between  command  and  control  and  both  levels 
of  mission  accomplishment. 

Relating  Criteria  to  Real  Combat 

Whatever  criteria  are  used,  the  tie  to  combat  outcome  must  be 
retained.  There  should  be  a  great  deal  of  commonality,  at  least  in 
principle,  between  the  problem  of  evaluating  the  command  and  control 
processes  in  a  computer-aided  evaluation  exercise  and  evaluating 
command  and  control  processes  in  an  actual  combat  situation.  In  each 
case,  it  makes  sense  to  ask  what  role  command  and  control  played  in 
the  resultant  outcome,  and  why,  and  whether  variations  in  command  and 
control  would  have  produced  significantly  different  outcomes.  The 
value  of  an  exercise  as  an  evaluative  tool,  after  all,  is  heavily 
dependent  on  its  ability  to  simulate  a  real  combat  environment  and  to 
reflect  the  impact  of  command  and  control  processes  in  that 
environment. 

There  are  also  important  differences  between  an  exercise 
environment  and  a  real  combat  environment  which  need  to  be  considered 
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in  designing  and  conducting  command  and  control  evaluations. 

Evaluation  of  most  real  combat  is  retrospective,  taking  place  after 
the  combat  is  over  using  data  which  happens  to  be  available  at  that 
time.  Evaluation  of  exercises,  on  the  other  hand,  can  be  structured 
and  thought  through  in  advance,  allowing  a  degree  of  planning  and 
rigor  not  achievable  with  real  combat.  Data  collection  can  likewise 
be  planned  in  advance,  and  the  exercise  can  be  instrumented  and 
monitored  to  a  degree  which  would  be  impossible  with  real  combat. 

Real  combat  is  a  one-time  affair,  with  only  the  combat  as  it 
occurred  to  look  at.  Exercises,  on  the  other  hand,  can  be  repeated 
and  systematically  varied  to  provide  something  akin  to  multiple 

samples  of  the  same  basic  combat  process.  The  structure  and  unfolding 

/ 

of  the  scenario  and  the  combat  problems  faced  by  the  C2  Components  can 
be  optimized  for  the  study  of  the  particular  evaluation  question  of 
interest,  which  can  never  be  done  in  actual  combat.  The  Evaluation 
Unit  and  the  evaluations  it  conducts  should  be  designed  in  such  a  way 
as  to  take  maximal  advantage  of  these  differences  in  the  evaluation 
process . 

But  the  similarities  should  not  be  ignored,  and  the  fact  that  the 
exercise  is  only  an  artificial  proxy  for  a  real  combat  environment 
should  not  be  forgotten.  The  basic  questions  are  the  same  in  either 
case,  and  an  approach  to  evaluation  which  does  not  make  sense  when 
applied  to  real  combat  probably  should  not  be  applied  to  exercise 
evaluation  either.  A  good  test  to  apply  to  any  evaluation  plan 
proposed  for  an  exercise  evaluation,  perhaps,  is  to  ask  what  kind  of 
sense  the  evaluation  concepts  being  employed  (measures  of  merit,  etc.) 
would  make  when  applied  to  a  real  combat  situation. 


EVALUATION  USING  THE  CAEF 

The  CAEF  may  operate  in  a  number  of  evaluative  modes,  some  of 

which  are  familiar  and  traditional  and  others  of  which  are  not.  It 

may  be  used,  for  example,  to  perform  evaluations  of  command  and 

control  systems  or  procedures  based  on  the  running  of  a  single 

exercise,  as  is  sometimes  done  now  with  existing  manual  exercises.  In 

this  mode,  the  CAEF  not  only  provides  a  richer  and  more  responsive 

combat  environment  against  which  to  pit  the  C2  Components  than  does  a 

conventional  exercise,  it  also  provides  a  capability  to  measure  and 

record  what  goes  on  during  the  exercise  to  a  degree  of  detail  which  is 

not  possible  during  a  manual  exercise,  and  to  reduce  the  resulting 

data  to  maiageable  form  and  produce  usable  evaluation  analyses  from 
% 

it. 

Systematic  variations  may  be  performed  on  a  particular  exercise 
or  exercise  theme,  using  a  "mixed  medium"  of  computer-aided  manned 
exercises  and  fully  computerized  simulation  runs.  Time  may  be  stopped 
and  restarted,  slowed  down  or  speeded  up.  Selected  portions  of  the 
exercise  may  be  rerun  with  some  variation,  either  with  manned  C2 
Components  or  as  a  pure  simulation  with  necessary' command  and  control 
inputs  supplied  by  the  evaluators.  This  mode  is  analogous  to  (though 
far  richer  in  possibilities  than)  making  multiple  runs  of  a  computer 
simulation  for  sensitivity  analysis,  but  has  no  counterpart  in  manned 
exercises  as  they  are  currently  conducted. 

The  facility  may  also  be  used  to  collect  and  compare  similar 
situations  occurring  in  a  number  of  exercises  over  time,  providing 
systematic  longitudinal  evaluation  of  a  sort  not  currently  available. 
Because  of  the  ability  of  the  computerized  facility  to  retain  and 


retrieve  detailed  descriptions  of  past  exercises  according  to  a  wide 
variety  of  criteria,  this  type  of  analysis  will  be  feasible  to  an 
extent  not  possible  without  such  capabilities. 

Other  modes  of  operation  are  likely  to  be  discovered  and 
developed  as  the  facility  is  utilized.  The  important  point  here, 
perhaps,  is  that  this  kind  of  facility  provides  capabilities  for 
comparison  and  evaluation  of  a  sort  which  do  not  currently  exist,  and 
the  use  of  these  capabilities  over  time  is  likely  to  bring  about 
fundamental  changes  in  our  concepts  of  what  exercises  and  evaluations 
are  and  how  they  should  be  interconnected.  How  fully  the  potential  of 
these  capabilities  is  realized,  of  course,  will  depend  on  the  CAEF 
staff  responsible  for  their  utilization.  This  strengthens  the  point 
made  earlier  that  the  ultimate  responsibility  for  evaluation  rests 
with  people  and  not  with  methods  or  computers,  which  should  properly 
be  seen  as  aids  and  tools  for  the  people. 

THE  EVALUATION  UNIT 

Within  the  CAEF,  evaluation  will  be  performed  by  the  Evaluation 
Unit.  This  Unit  will  be  responsible  for  preparing  an  evaluation 
design  prior  to  each  exercise,  collecting  necessary  data  during  the 
exercise,  and  analyzing  that  data  to  produce  evaluation  reports 
following  the  exercise.  The  activities  and  functions  of  the 
Evaluation  Unit  fall  into  two  distinct  classes--those  associated  with 
individual  exercises  and  those  associated  with  the  operation  of  the 
CAEF  as  a  long-term  ongoing  facility. 

Evaluation  Unit  activities  associated  with  a  particular  exercise 
start  long  before  the  exercise  itself  begins.  From  the  very  beginning 


of  the  exercise  cycle.  Evaluation  Unit  personnel  will  be  involved  in 
defining  and  laying  out  evaluation  goals  and  objectives,  and  in 
ensuring  that  the  developing  exercise  design  will  meet  those 
objectives.  During  the  pre-exercise  period,  they  will  interact  with 
exercise  designers  from  the  exercise  driver  staff  as  well  as  with 
others  from  staff  and  operational  units  concerned  one  way  or  another 
with  the  upcoming  exercise  and  with  the  evaluation  scheduled  to  be 
performed. 

If  new  equipment  or  systems  are  being  evaluated,  for  example, 
coordination  will  be  required  between  the  evaluation  unit  and  the  Air 
Force  Systems  Command  (AFSC)  component  responsible  for  the  new  system. 
If  the  evaluation  is  focused  on  new  tactics  or  procedures, 
coordination  with  appropriate  operational  units  may  be  needed.  If 
capabilities  to  respond  in  a  particular  geographic  area  are  the 
subject  of  the  investigation,  there  may  be  considerable  interaction 
with  the  specified  command  responsible  for  that  area. 

One  of  the  major  activities  of  the  Evaluation  Unit  during  the 
pre-exercise  period  will  be  development  of  an  evaluation  plan  for  the 
upcoming  exercise.  The  evaluation  plan  must  include  a  preliminary 
model  of  the  exercise  conflict  to  serve  as  a  basis  for  the  development 
of  the  exercise  description  on  which  the  later  evaluation  will  rest, 
as  well  as  specification  of  the  observation  points  (both  manual  and 
automated)  required  for  the  exercise  and  the  data  to  be  gathered  at 
each.  The  plan  should  also  outline  the  structure  of  the  post-exercise 
analysis,  including  the  measures  of  effectiveness  to  be  used  and  the 
kinds  of  conclusions  expected. 


-49- 


Command  and  control  systems  or  processes  cannot  be  given  a  fair 
evaluation  in  an  exercise  unless  the  personnel  using  them  have  a 
degree  of  familiarity  and  experience  with  those  systems  comparable  to 
that  which  would  be  expected  under  field  conditions.  It  will  be  the 
responsibility  of  the  evaluation  unit  to  determine  how  much  pre¬ 
exercise  training  and  familiarization  the  players  from  the  C2 
Components  should  have  with  any  new  procedures  or  systems,  and  to 
monitor  that  training  in  order  to  ensure  that  adequate  levels  of 
proficiency  are  reached  prior  to  the  exercise. 


Fig.  7  —  Relationship  of  the  evaluation  unit 
to  the  exercise  conflict  structure 


During  the  play  of  the  exercise,  the  activities  of  the  Evaluation 
Unit  will  focus  primarily  on  monitoring  and  data  collection.  The 
relationship  between  the  Evaluation  Unit  and  the  other  exercise 
components  is  shown  in  Fig.  7.  The  Evaluation  Unit,  in  effect,  sits 
on  the  top  of  the  other  components,  observing  and  drawing  data  from 
them.  In  addition,  there  will  be  sufficient  overlap  between  the 
Evaluation  Unit  and  the  Control  Module  to  ensure  that  the  exercise  is 
managed  in  a  manner  consistent  with  the  evaluation  objectives  and  that 
those  objectives  are  being  met.  Data  collection  will  proceed  in 
accordance  with  the  previously  developed  exercise  plan,  though  the 
human  observers  in  the  Evaluation  Unit  should  remain  alert  for 
significant  factors  or  events  not  previously  anticipated.  The  fact 
that  most  of  the  data  collection  is  preplanned  should  by  no  means 
preclude  serendipitous  observation.  On  the  contrary,  serendipitous 
observations  by  expert  observers  should  be  one  of  the  most  important 
sources  of  information  for  the  Evaluation  Unit. 

Data  collection  will  continue  into  the  wind-down  period 
immediately  following  the  exercise,  with  debriefings  of  participants 
and  observers  alike.  Some  of  this  debriefing  should  be  highly 
structured,  e.g.,  administration  of  structured  questionnaires  designed 
earlier  as  part  of  the  evaluation  plan,  while  some  of  it  should  be 
much  more  loosely  structured,  e.g.,  open-ended  interviews  and 
discussions . 

Some  analysis  may  begin  during  the  running  of  the  exercise,  but 
the  bulk  of  the  analytical  effort  will  take  place  after  the  completion 
of  the  exercise  itself,  culminating  in  the  production  of  whatever 
reports,  briefings,  or  other  evaluation  products  are  called  for  by  the 
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exercise  plan  or  are  otherwise  appropriate  for  that  particular 
exercise. 

In  addition  to  evaluations  associated  with  particular  individual 
exercises,  the  Evaluation  Unit  will  have  an  ongoing  responsibility  for 
organizing  and  maintaining  the  cumulative  database  derived  from  all 
exercises  run  at  the  CAEF  and  for  performing  comparative  evaluations 
across  exercises  using  that  database.  At  the  same  time,  the 
Evaluation  Unit  will  also  have  responsibilities  for  evolving  the  CAEF 
itself  and  developing  and  refining  concepts  for  using  it  as  an 
evaluative  tool. 

THE  PROCESS  OF  EVALUATION 

In  an  area  as  nonrigorous  as  command  and  control,  evaluation  is 
ultimately  subjective,  in  the  sense  that  it  is  something  which  takes 
place  in  the  minds  of  the  people  involved.  People  decide, 
individually  and  collectively,  on  the  basis  of  the  information  before 
them,  whether  particular  systems,  procedures,  etc.,  are  worthwhile  or 
not;  whether  changes  being  contemplated  will  improve  or  degrade  the 
total  process;  how  much  gain  can  be  expected  from  a  proposed 
improvement;  and  other  similar  evaluative  questions.  Quantitative 
measurement,  analytic  methodology,  computer  modeling,  and  the  like, 
are  useful  tools  in  this  process,  both  in  structuring  and  giving 
meaning  to  the  information  on  which  evaluation  is  to  be  based  and  in 
providing  a  vehicle  for  the  communication  of  information  and 
conclusions.  They  are  not,  however,  the  ultimate  basis  for  important 
substantive  conclusions.  These  remain  the  responsibility  of  human 
analysts  and  evaluators.  (Even  when  conclusions  appear  to  be  based 
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solely  on  the  application  of  formal  analytic  techniques  to  objective 
empirical  data,  that  remains  true.  Someone  chose  to  use  those 
techniques  and  to  apply  them  to  that  data,  and  those  choices  were 
ultimately  subjective.  The  conclusions  reached  are  simply  logical 
extensions  and  implications  of  those  choices.  See  [3]  for  further 
discussion  of  these  issues . ) 

Related  back  to  the  Evaluation  Unit,  this  means  that  the  human 
evaluators  in  the  Unit  are  the  source  of  the  evaluations  it  produces, 
not  the  formal  models  and  methods  they  employ,  and  certainly  not  the 
computer  which  implements  those  models  and  methods.  Quantification, 
in  particular,  should  not  be  approached  as  an  end  in  itself,  but 
rather  as  a  means  to  an  end.  It  must  be  guided  by  and  interpreted 
with  careful  and  considered  qualitative  judgment  within  the  evaluation 
process  if  its  potential  value  is  to  be  fully  realized. 

One  thing  which  the  Evaluation  Unit  will  require  is  a  model  of  or 
paradigm  for  the  process  of  evaluation  around  which  the  activities  of 
the  Unit  can  be  organized.  One  possible  paradigm  which  might  serve 
that  purpose  is  outlined  below.  It  is  not  the  only  one  imaginable, 
and  perhaps  not  the  best,  but  it  seems  to  provide  a  reasonable  place 
to  start.  We  will  focus  primarily  on  the  problem  of  evaluating 
command  and  control  in  a  particular  conflict  or  exercise  situation, 
extending  the  ideas  developed  to  problems  of  comparison  across 
situations,  or  across  systems  in  the  same  situation,  as  appropriate. 

The  process  of  evaluating  the  contribution  of  command  and  control 
in  a  particular  exercise  situation  is  conceptually  similar  regardless 
of  whether  the  evaluator  considered  is  a  single  individual  performing 
an  informal  subjective  process  in  his  head  or  a  specialized  component 
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(e.g.,  the  Evaluation  Unit)  of  a  large  organization  performing  a 
complex  and  highly  formalized  evaluation.  It  is  these  conceptual 
similarities  which  we  wish  to  identify,  understand  and  use  as  a  basis 
for  the  structure  and  operation  of  the  Evaluation  Unit. 


Fig.  8  —  The  evaluation  process 

Evaluation  Requires  Description 

The  nature  of  the  evaluation  process  is  depicted  in  Fig.  8.  The 


evaluator  (single  individual  or  Evaluation  Unit)  creates  an  image  or 
description  of  the  exercise  that  has  occurred  (we  will  speak  of  the 
exercise  in  past  tense,  recognizing  that  some  of  this  will  take  place 
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as  the  exercise  is  actually  occurring)  which  will  serve  as  the  basis 
for  the  evaluation  process.  This  describing  process  depends  heavily, 
of  course,  on  preliminary  activities  which  take  place  in  advance  of 
the  exercise,  including  pre-exercise  planning  for  what  the  description 
will  be  like  and  how  it  will  be  built.  Within  that  description,  the 
evaluator  identifies  attributes  of  the  outcome  which  he  considers 
important,  and  characterizes  the  links  between  the  command  and  control 
elements  he  wishes  to  evaluate  and  those  outcome  attributes.  He 
attaches  a  valuation  to  the  outcome  attributes  identified,  and  uses 
the  links  to  infer  valuation  back  to  the  command  and  control  elements. 

Even  from  this  relatively  crude  conceptual  model  a  number  of 
interesting  and  useful  observations  can  be  drawn.  Note,  for  example, 
the  fact  that  the  evaluator  works  with  a  description  of  the  conflict 
and  not  with  the  conflict  itself,  and  that  this  description  will 
always  be  partial  and  incomplete.  This  implies  that  the  process  of 
forming  this  description  is  itself  an  important  part  of  the  evaluation 
process,  which  deserves  careful  and  explicit  consideration  in  its  own 
right. 

It  also  implies  that  when  we  go  beyond  evaluation  of  a  single 
exercise  and  consider  cross  comparison  between  a  number  of  different 
exercises,  the  degree  of  comparability  between  the  descriptions  which 
are  available  of  the  different  exercises  to  be  compared  will  be 
important.  For  this  reason,  the  nature  of  the  description  forming 
process  per  se  needs  more  attention  in  designing  the  CAEF  as  an 
ongoing  facility  than  might  be  necessary  for  the  evaluation  of  an 
individual  exercise. 

The  description  forming  process  is  composed  of  two  principal 
components,  data  collection  and  description  generation  from  the 


collected  data.  In  the  case  of  the  individual  evaluator,  the 
description  is  his  subjective  image  of  the  exercise,  and  the  processes 
of  data  collection  and  description  generation  go  on  simultaneously  as 
he  observes  the  exercise  and  forms  impressions  of  what  is  happening 
and  why.  He  collects  data  through  his  eyes  and  ears,  and  those 
incoming  data  contribute  to  and  modify  his  description  of  the  ongoing 
exercise.  But  at  the  same  time,  the  data  he  collects  and  the  way  he 
interprets  those  data  are  themselves  conditioned  by  his  present 
description- -by  his  expectations  and  impressions  of  what  is  occurring 
and  of  what  is  important  to  observe  and  pay  attention  to. 

In  the  case  of  the  Evaluation  Unit  the  description  is  a  more 
formal  summary  and  description  of  what  went  on  during  the  course  of 
the  exercise.  The  situation  at  first  glance  appears  simpler  than  in 
the  case  of  the  individual,  with  most  of  the  data  collection  occurring 
during  the  course  of  the  exercise  and  most  of  the  description 
generation  occurring  after  the  exercise  as  those  data  are  summarized 
and  analyzed.  But  even  here  the  individual  members  of  the  Evaluation 
Unit  are  forming  descriptions  and  opinions  as  the  exercise  goes  on, 
and  these  will  contribute  to  the  collective  description  of  the 
Evaluation  Unit  on  which  the  exercise  evaluation  will  be  based.  There 
may  also  be  some  post-exercise  data  collection,  such  as  debriefing  of 
players.  Even  the  individual  phenomenon  of  prior  expectations 
influencing  the  data  collection  process  has  a  parallel  in  group 
evaluation,  in  that  prior  expectations  of  how  the  exercise  would  go 
and  what  would  be  important  to  observe  (which  can  be  thought  of  as  a 
prior  description)  will  have  influenced  the  data  collection  design. 
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Though  the  describing  process  appears  to  take  place  primarily 
during  and  after  the  exercise,  it  rests  heavily  on  activities  taking 
place  much  earlier.  In  the  case  of  the  individual  evaluator  these 
include  the  earlier  acquisition  of  the  background  and  experience 
necessary  to  filter  and  interpret  what  he  sees,  as  well  as  the 
preparation  he  puts  into  getting  ready  for  this  particular  exercise. 

In  the  case  of  the  Evaluation  Unit,  it  includes  these  things  as  they 
relate  to  the  individual  members  of  the  Unit,  as  well  as  the 
organizational  planning  and  preparation  put  into  the  development  of 
the  Evaluation  Plan  for  the  exercise.  Careful  and  thoughtful  prior 
planning  is  particularly  necessary  if  the  descriptions  of  individual 
exercises  are  to  have  enough  in  common  to  support  good  longitudinal 
analyses  across  many  exercises. 

The  Logic  of  Evaluation 

Identifying  the  outcome  attributes  of  interest  and  linking  them 
to  the  command  and  control  elements  under  investigation  lies  at  the 
heart  of  the  evaluation  process.  In  some  cases,  this  linkage  may  be 
made  in  a  single  direct  step,  while  in  others  (and  this  will  probably 
be  more  frequent),  it  will  be  indirect  and  depend  on  one  or  more 
intervening  variables.  In  some  cases,  it  may  be  possible  to  identify 
and  describe  the  link  very  directly,  while  in  others  it  can  only  be 
assumed  to  exist  and  its  characteristics  measured  by  inference.  (This 
is  what  is  being  done,  for  example,  when  statistical  techniques  are 
applied  to  a  sample  of  several  similar  exercises.) 

In  all  cases,  however,  the  underlying  logic  of  evaluation  is  the 
same.  Command  and  control  is  assumed  to  exert  an  influence  on  the 


outcome  of  the  exercise,  as  indicated  by  the  right -pointing  arrow  in 
Fig.  8,  which  can  be  measured  or  otherwise  inferred  from  the  exercise 
description.  It  is  also  assumed  that  the  outcome  itself  can  be  valued 
in  a  meaningful  way.  This  valuation  on  the  outcome  can  then  be 
transferred  back  to  a  valuation  on  the  command  and  control  process 
which  produced  that  outcome,  as  indicated  by  the  left-pointing  arrow 
in  Fig.  8. 

For  the  full  potential  of  the  CAEF  to  be  realized,  an  evaluation 
paradigm  that  can  guide  the  evaluation  function  across  exercises  over 
time,  as  well  as  the  evaluation  of  individual  exercises,  will  be 
required.  The  final  form  of  this  paradigm  can  only  evolve  with 
experience,  and  must  be  developed  by  the  Evaluation  Unit  itself. 
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VI.  DEVELOPMENTAL  ISSUES 

In  the  preceding,  we  have  outlined  the  philosophy  and  general 
structure  for  a  CAEF.  That  discussion  necessarily  omitted 
consideration  of  a  number  of  practical  issues  related  to  the 
establishment  of  such  a  facility,  such  as  location,  computer  support, 
and  manning.  We  now  briefly  consider  these  issues.  Remember  that  we 
are  thinking  of  the  CAEF  not  simply  as  a  facility  for  conducting  a 
series  of  conceptually  distinct  one-time  exercises,  but  as  an  ongoing 
and  evolving  instrument  for  the  exploration  and  evaluation  of  command 
and  control  systems  and  processes. 

LOCATION 

In  principle,  the  CAEF  could  be  thought  of  as  a  completely  new 
facility,  built  from  the  ground  up  at  any  location  where  suitable  real 
estate  is  available.  In  practice,  the  choices  available  are  not 
unlimited,  and  the  CAEF  should  be  developed  in  conjunction  with 
existing  Air  Force  training  and  evaluative  activities  which  it  can 
draw  from  and  mutually  support.  The  most  obvious  location  for  such  a 
facility  would  be  at  Hurlburt  Field,  Fla.,  as  part  of  the  Tactical  Air 
Warfare  Center  (TAWC).  The  exercise  facility  itself  should  probably 
be  part  of  the  C3I  Complex  now  developing  at  Hurlburt  Field. 


COMPUTER  SUPPORT 

The  CAEF  will  require  extensive  computer  support,  particularly 
during  the  actual  conduct  of  exercises,  and  a  number  of  options  exist 
for  providing  that  support.  The  facility  might  be  provided  with  one 
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large  computer  mainframe  to  support  all  its  activities,  or  it  might 
have  several  smaller  ones.  These  could  be  dedicated  computers,  or 
could  be  shared  with  other  users.  And  if  the  latter,  they  could  be 
located  at  the  CAEF,  or  at  remote  locations.  These  choices  obviously 
affect  the  operations  of  the  facility,  and  its  cost. 

These  questions  interact  strongly  with  some  of  the  Exercise 
Driver  design  issues  raised  earlier.  If  the  Driver  has  a  relatively 
fixed  structure  based  around  a  single  large  combat  model,  then  a 
single  dedicated  mainframe  capable  of  handling  that  model  in  real  time 
would  appear  to  be  the  natural  choice.  On  the  other  hand,  if  the 
Driver  is  highly  modular,  it  might  be  just  as  efficient  to  internet  a 
number  of  different  computers,,  each  handling  different  modules, 
through  a  central  controlling  computer. 

It  does  seem  clear  that  the  CAEF  will  require  some  dedicated 
computational  capability,  at  least  enough  to  support  the  functions  of 
evaluation  and  exercise  development  on  an  ongoing  basis  (between  as 
well  as  during  exercises).  It  is  less  clear,  however,  that  the  CAEF 
needs  to  possess  the  much  larger  computational  capability  required  to 
support  an  exercise  in  real  time,  when  this  capability  is  likely  to  be 
required  only  relatively  infrequently.  If  the  system  were  modular, 
the  controlling  computer  might  belong  to  the  CAEF  and  serve  to  support 
CAEF  functions  between  exercises ,  with  other  computers  netted  into  the 
facility  as  required  during  exercises.  One  consideration  favoring 
this  kind  of  architecture  is  that  it  would  allow  the  Exercise  Driver 
to  draw  from  and  utilize  some  of  the  same  operational  systems  which 
would  support  command  and  control  functions  during  actual  combat. 
Another  is  the  fact  that  many  of  the  systems  being  evaluated  will  be 
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computer  i  zed  systems,  which  the  CAEF  will  need  to  be  able  to  interact 
with  and  integrate  into  its  operation. 

Another  consideration  which  might  favor  providing  the  CAEF  with 
one  or  more  small  computers  rather  than  a  single  large  mainframe  is 
the  fact  that  the  C3I  Complex  appears  to  be  the  end  destination  for  a 
number  of  computers  (largely  PDP  11/ 70s)  which  the  Air  Force  has 
purchased  and  used  for  a  variety  of  developmental  programs.  One  or 
more  of  these  computers  might  easily  be  made  available  for  the 
facility  without  the  cost  of  purchasing  a  new  mainframe. 


MODES  OF  OPERATION 

During  the  early  phases  of  its  operation,  the  CAEF  would  probably 
operate  in  a  mode  similar  to  the  way  Blue  Flag  exercises  are  currently 
conducted.  That  is  to  say  that  it  would  conduct  exercises  on  a 
regularly  scheduled  basis,  each  exercise  more  or  less  independent  of 
the  others.  The  individual  exercises  themselves  would  initially  be 
straightforward  computerized  extensions  of  the  kinds  of  manual  Command 
Post  Exercises  run  today. 

As  the  CAEF  was  used  and  experience  was  gained  with  it,  we  would 
expect  it  to  evolve  a  very  different  style  of  operation  from  anything 
which  exists  today.  New  forms  of  exercises  and  of  man/machine 
analysis  should  evolve,  combining  aspects  of  exercises  and  aspects  of 
computer  modeling  analysis  in  a  manner  appropriate  to  the  hybrid 
facility  which  the  CAEF  will  become.  Some  of  the  possibilities  have 
been  briefly  alluded  to  earlier. 

Taking  advantage  of  these  new  possibilities  will  require  the 
development  of  new  concepts  and  procedures,  and  we  would  expect  this 
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development  to  occupy  a  major  place  In  the  operation  of  the  CAEF.  The 
CAEF  will  have  to  work  out  its  own  style  of  operation  as  it  goes,  and 
evolve  ways  of  utilizing  the  capabilities  it  provides. 

The  CAEF  will  have  the  inherent  capability  to  collect  and  process 
masses  of  data  for  each  exercise.  Effectively  utilizing  this 
capability  will  require  more  elaborate  description  and  documentation 
(and  the  ability  to  retrieve  selected  data)  of  individual  exercises 
than  is  done  at  present,  and  ways  to  do  this  will  have  to  be  developed 
by  the  CAEF  itself.  Once  in  place,  however,  they  should  pay 
significant  dividends  in  the  productivity  of  the  facility.  In 
addition,  the  development  of  the  necessary  concepts  and  procedures  for 
describing  individual  exercises  and  making  comparisons  between  them 
should  itself  enhance  Air  Force  understanding  of  the  role  of  command 
and  control  in  conflict. 

A  series  of  standard  scenarios  might  be  developed,  for  example, 
which  could  be  used  repeatedly  in  different  exercises  against 
different  command  and  control  process  alternatives.  If  each  of  these 
exercise  runs  were  documented  in  a  standard  way,  then  replication 
could  be  obtained  across  runs  more  economically  than  would  be  possible 
if  all  needed  replications  had  to  be  run  separately  each  time  an 
evaluation  requiring  replication  was  made.  If  a  new  sensor  or 
communications  system  were  to  be  evaluated,  past  runs  of  the  same 
scenario  in  which  that  system  had  not  been  played  could  provide  the 
control  cases  against  which  its  effects  could  be  compared.  Command 
and  control  vulnerability  could  be  explored  with  repeated  runs  of  the 
same  or  similar  exercises  in  which  different  command  and  control 
elements  were  attacked  and  put  out  of  action. 
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A  further  base  for  replication  and  comparison  could  be  obtained 
by  identifying  and  keeping  track  of  key  situations  or 
situation/response  combinations  occurring  more  than  once  in  an 
individual  exercise.  A  database  of  low-level,  action-effect 
measurements  could  be  built  up,  which  could  then  be  aggregated  as 
appropriate  in  different  ways.  Suppose,  for  example,  it  were 
necessary  to  evaluate  the  effectiveness  of  a  new  sensor  system  which 
provided  timely  identification  and  targeting  of  second  echelon  movers. 
Evaluating  the  effectivness  of  the  system  in  terms  of  impact  on 
outcomes  might  be  made  much  easier  if  the  exercise  measurement  of  the 
performance  of  the  system  (in  terms  of  fraction  of  movers  identified) 
could  be  combined  with  estimates  made  over  a  number  of  past  exercises 
of  effectiveness  as  a  function  of  fraction  of  targets  known. 

MANNING  AND  EQUIPPING  THE  CAEF 

All  this  implies  a  need  for  a  high-quality  innovative  staff  for 
the  CAEF,  particularly  during  its  early  phases,  and  for  a  higher 
degree  of  staff  continuity  than  is  sometimes  the  case  in  military 
organizations.  One  way  in  which  this  continuity  can  be  achieved  is 
through  mixed  military/civilian  staffing,  with  the  civilians  providing 
some  of  the  continuity  and  institutional  memory.  For  the  military 
personnel,  tours  should  probably  be  fairly  long  and  staggered  enough 
so  that  large  staff  turnover  during  a  short  period  is  avoided.  In 
particular,  personnel  initially  assigned  to  the  CAEF  should  have 
staggered  tour  lengths,  in  order  to  avoid  the  problem  of  suddenly 
having  a  large  fraction  of  the  staff  turn  over  at  the  end  of 
everybody's  first  tour. 
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It  seems  likely  that  the  cadre  associated  with  the  Exercise 
Driver--the  people  responsible  for  putting  together  and  running  the 
exercises --should  be  permanent  party  staff.  This  would  include  not 
only  scenario  designers  and  exercise  management  personnel,  but  also 
the  operations  analysts,  computer  analysts,  and  programmers 
responsible  for  creating  and  maintaining  the  computer  models  and 
programs  necessary  for  the  Exercise  Driver.  This  permanent  party 
might  or  might  not  need  to  be  augmented  during  the  exercise  by  TDY 
personnel,  depending  on  the  nature  of  the  exercise  and  the  manpower 
required  by  the  Exercise  Driver. 

In  the  case  of  the  Evaluation  Unit,  it  seems  plausible  that  a 
mixture  of  permanent  party  and  TDY  personnel  will  be  required.  Much 
of  the  advantage  of  the  CAEF  as  an  evaluation  vehicle  comes  about  as  a 
result  of  its  ability  to  create  and  exploit  a  growing  body  of  data 
about  the  impacts  of  command  and  control  on  exercise  outcomes,  and  to 
utilize  appropriate  segments  of  past  data  in  the  design  and  evaluation 
of  each  new  exercise.  This  imposes  a  requirement  for  a  highly 
competent  permanent  party  staff  to  provide  the  continuity  necessary  to 
evolve  the  facility  on  an  ongoing  basis  and  to  provide  a  knowledgeable 
and  experienced  cadre  at  each  exercise.  It  is  not  reasonable  to 
expect  this  staff  to  be  large  enough  or  experienced  enough  in  the  wide 
variety  of  command  and  control  questions  likely  to  be  evaluated  using 
the  CAEF,  however,  to  fully  man  the  Evaluation  Unit  during  exercises. 
They  would  probably  be  supplemented  for  each  exercise  with  experts  on 
the  systems  or  issues  under  evaluation  in  that  exercise,  and  might 
draw  additional  observers  and  data  collectors  from  other  sources  as 


needed . 
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There  are  a  variety  of  options  for  manning  and  equipping  the  C2 
Components,  all  of  which  have  both  merits  and  weaknesses.  There  are 
reasonable  arguments  both  for  and  against  a  permanently  designated 
exercise  team  manning  the  C2  Components  on  a  regular  basis  who  are 
expertly  qualified  in  its  operation  and  functioning.  On  the  one  hand, 
such  a  team  would  minimize  the  need  for  pre-exercise  familiarization 
with  the  facility  itself  and  would  reduce  the  variation  in  performance 
attributable  to  variations  in  manning.  On  the  other  hand,  such  a  team 
might  in  time  become  too  familiar  with  the  CAEF,  and  begin  "playing 
against  the  Exercise  Driver"  rather  than  behaving  as  they  would  in  a 
real  conflict  environment.  Using  the  same  team  all  the  time  also 
complicates  rerunning  minor  variations  of  the  same  scenario,  since 
having  played  that  scenario  in  the  past  would  affect  their  responses 
to  it. 

Other  manning  sources  which  should  be  considered  might  include 
the  use  of  personnel  from  operational  TACS  units  or  the  assembling  of 
ad  hoc  C2  Components  teams  for  exercises  for  which  that  seemed 
appropriate.  In  exercises  involving  the  employment  of  new  or 
experimental  systems,  manning  might  also  include  personnel  already 
familiar  with  the  particular  system  being  tested.  We  would  expect 
that  the  appropriate  mix  of  these  manning  sources,  and  procedures  for 
selecting  the  right  one  for  each  particular  exercise,  would  evolve 
over  time  with  the  evolution  of  the  CAEF  itself. 

With  respect  to  equipment,  it  seems  clear  that  the  CAEF  should 
provide  the  basic  equipment  and  facilities  required  by  the  C2 
Components.  In  order  to  provide  maximum  flexibility  in  utilizing  the 
CAEF,  however,  its  operation  should  not  be  irrevocably  tied  to  the  use 


k 


of  this  equipment.  Rather,  it  should  be  able  to  feed  and  receive 
information  from  C2  elements  via  standard  C2  communications  channels , 
so  that  operational  elements  can  be  exercised  using  their  own 
facilities  and  equipment  as  well  as  those  of  the  CAEF.  In  addition, 
the  CAEF  should  have  sufficient  flexibility  to  accommodate  and  adapt 
to  new  systems  and  equipment  played  in  the  C2  Components. 

THE  EVOLUTIONARY  PROCESS 

The  CAEF  will  not  be  created  in  a  workable  finished  form  in  a 
single  stroke,  but  will  evolve  through  a  series  of  updates,  changes, 
and  modifications  from  today's  existing  exercise  capabilities,  as 
outlined  in  Fig.  9.  The  structure  which  eventually  emerges  will  have 
many  general  features  in  common  with  what  exists  today,  but  will  also 
differ  from  what  now  exists  in  significant  and  important  ways. 

At  present,  exercises  are  manually  supported  and  heavily  pre¬ 
scripted,  conducted  primarily  for  training  with  whatever  evaluation  is 
performed  added  on  in  a  fairly  ad  hoc  way.  The  world  outside  the 
manned  C2  elements  is  handled  by  a  manual  control  team  working  from  a 
canned  scenario  which  permits  only  minor  changes  from  a  pre-scripted 
time  history  of  conflict  independent  of  the  actions  taken  by  the  C2 
elements.  Waiting  in  the  wings  for  test  or  evaluation  there  are 
always  a  variety  of  C2  systems,  in  various  stages  of  development. 

Some  of  these  will  be  inserted  in  each  exercise  to  see  how  they  work 
out.  During  the  course  of  the  exercise,  informal  evaluations, 
perceptions,  and  policy  implications  are  evolved.  These  generally  are 
mentioned  and  discussed  at  the  exercise  debriefing,  but  most  slip  away 


and  only  a  small  portion  of  the  potential  knowledge  which  might  be 
gained  from  the  exercise  is  systematically  retained. 


Fig.  9  —  Evolution  of  CAEF 


In  a  mature  and  functional  CAEF,  the  manual  control  team  will 
have  been  replaced  by  a  man-machine  exercise  driver  and  the  canned 
scenario  will  have  been  replaced  by  a  dynamic  scenario  responsive  to 
the  action  and  decisions  of  the  C2  Components.  New  command  and 


control  systems  and  components  will  continue  to  require  test  and 
evaluation,  and  evaluation  will  be  more  of  a  primary  exercise 
objective  than  is  currently  the  case.  An  ongoing  Evaluation  Unit  will 
have  been  added  which  will  design  and  conduct  evaluations  in  a  far 
more  rigorous  and  consistent  manner  than  is  possible  today.  These 
evaluations  and  the  knowledge  gained  from  them  will  be  retained  and 
utilized  in  the  design  and  evolution  of  more  effective  and  capable 
command  and  control  systems  for  U.S.  tactical  air  forces. 

Getting  from  here  to  there  will  be  a  complex  and  difficult  task, 
requiring  choices  at  a  number  of  branch  points  as  the  facility 
evolves.  The  eventual  path  can  be  only  dimly  perceived  at  present. 
What  we  hope  to  do  is  to  cast  a  little  more  light  on  that  path  and 
point  out  good  directions  to  start  out,  recognizing  that  we  cannot 
supply  a  complete  roadmap  at  the  outset. 
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